neeto-email-delivery-nano

The neeto-email-delivery-nano acts as the source of truth for the new nano's structure, configs, data etc..

Contents

1. Development with Host Application
   • Engine
     • Installation
     • Usage
   • Frontend package
     • Installation
     • Instructions for development
     • Usage
2. Automation testing endpoints
3. Instructions for Publishing

Development with Host Application

Engine

Installation

1. Add this line to your application's Gemfile:

    source "NEETO_GEM_SERVER_URL" do
      # ..existing gems
   
      gem 'neeto-email-delivery-engine'
    end

2. And then execute:

   bundle install

3. Add this line to your application's config/routes.rb file

    mount NeetoEmailDeliveryEngine::Engine, at: "/neeto_email_delivery"

4. Add required migrations in the db/migrate folder. Run the following commands to copy the migrations from the engine to the host application.

   bundle exec rails neeto_email_delivery_engine:install:migrations
   bundle exec rails db:migrate

5. Include NeetoEmailDeliveryEngine::DeliveryMethodSelector in the host's application_mailer.rb

class ApplicationMailer < NeetoCommonsBackend::ApplicationMailer
+  include NeetoEmailDeliveryEngine::DeliveryMethodSelector

  # ...
end

6. Include NeetoEmailDeliveryEngine::HasEmailIntegrations in the host's owner model and override the `email_delivery_redirect_url` and `get_organization_id` method

```diff
class Form < ApplicationRecord
# ...
+  include NeetoEmailDeliveryEngine::HasEmailIntegrations
# ...

  def email_delivery_redirect_url
    # "/admin/..."
  end

  def get_organization_id
    # organization_id
  end

end

7. Add config/initializers/neeto_email_delivery_engine.rb and add the owner class. Example:

NeetoEmailDeliveryEngine.owner_class = "Form"

8. Override the load_owner private method for the base controller.

# app/overrides/controllers/neeto_email_delivery_engine/base_controller_override.rb
NeetoEmailDeliveryEngine::Api::V1::BaseController.class_eval do

  private
    def load_owner
      # @owner = current_user
      # OR
      # @owner = @organization.forms.find(params[:owner_id])
    end
end

9. Override the send_disconnection_email and send_delivery_failure_email private methods -

• app/overrides/services/neeto_email_delivery_engine/base_email_service_override.rb

send_disconnection_email is called when the account itself has become unusable, for example when the credentials are rejected, and the account has already been disconnected by the time it runs.

send_delivery_failure_email is called when the provider permanently rejected a single email, for example when the message is over the size limit. The account is still connected and still works, so only that one email needs to be reported. There is no fallback send in this case: the provider gave a permanent refusal for this specific content, so re-sending it through Neeto's account would be refused again. Implement this hook, or a rejected email goes unreported and the owner never learns it was not delivered.

Both hooks receive every provider, so type is one of gmail, outlook or smtp in each.

reason means something different in each hook, and each hook always gets one shape, so no host method has to type-check it:

• send_disconnection_email always receives a Symbol classifying why the account was dropped — one of :authentication_failure, :authorization_revoked, :invalid_configuration or :unknown_error. Treat it as a lookup key and always keep a fallback branch, since more may be added.
• send_delivery_failure_email always receives the provider's own String, such as "552 5.3.4 Message size exceeds fixed maximum message size". It is meant to be shown to the account owner verbatim; it is their own provider describing their own message, not Neeto infrastructure detail.

How a failure is classified

Every provider sorts failures into the same three buckets. Only the middle one destroys the account.

| Bucket | SMTP | Gmail | Outlook | | --- | --- | --- | --- | | Transient — retried, account untouched | 4xx (Net::SMTPServerBusy), timeouts | 429, and 403 quota replies | 429, 5xx, transient token refresh failures | | Account unusable — disconnects | 53x (Net::SMTPAuthenticationError), invalid SSL config | 401, and 403 PERMISSION_DENIED / failedPrecondition / mail service not enabled | 401 and 403, invalid_grant on refresh | | Message rejected — reports, account survives | other 5xx (Net::SMTPFatalError) | other 4xx, e.g. an oversized message or a malformed recipient | NON_DISCONNECTION_ERROR_CODES, e.g. ErrorMessageSizeExceeded |

The default matters more than the table. An unrecognised failure is treated as a message-level problem, not an account-level one, because deleting a working integration is far more costly to the customer than reporting one failed email. Add to the account-level lists deliberately; do not let anything fall into them by accident.

NeetoEmailDeliveryEngine::BaseEmailService.class_eval do

  private

    def send_disconnection_email(integrable, organization_id, type, reason: nil)
      # Call your mailer logic
      # type: gmail, outlook, smtp
      # reason: why the account was disconnected, e.g. :authentication_failure

      # DisconnectionMailer
      #   .with(organization_id: organization_id)
      #   .send_email(user: integrable, type:, reason:)
      #   .deliver_later
    end

    def send_delivery_failure_email(integrable, organization_id, type, reason: nil)
      # Call your mailer logic
      # type: gmail, outlook, smtp
      # reason: the rejection reported by the provider, e.g.
      #   "552 5.3.4 Message size exceeds fixed maximum message size"

      # DeliveryFailureMailer
      #   .with(organization_id: organization_id)
      #   .send_email(user: integrable, type:, reason:)
      #   .deliver_later
    end
end

10. Add mailgun ingress config in application.rb

config.action_mailbox.ingress = :mailgun

11. Add :gmail and :outlook to omniauth_providers in config/initializers/extend_user_omniauth.rb.

12. The following env variables need to be added to the host application.

GOOGLE_OAUTH_CLIENT_ID
GOOGLE_OAUTH_CLIENT_SECRET
MICROSOFT_OAUTH_CLIENT_ID
MICROSOFT_OAUTH_CLIENT_SECRET
SPARKPOST_SENDING_DOMAINS_KEY

## The following are needed, but they should already be present in the host application. So ensure that they are present.
APP_LOCAL_TO_INTERNET_PROXY_URL
CONNECT_PRIVATE_KEY
CONNECT_PUBLIC_KEY
MAILGUN_INGRESS_SIGNING_KEY (needed for our custom logic to verify the Outlook account.)

13. In secrets.yml add

sparkpost:
  sending_domains_key: <%= ENV['SPARKPOST_SENDING_DOMAINS_KEY'] %>
  sending_domains_user_name: SMTP_Injection
  sending_domains_address: smtp.sparkpostmail.com

14. Ensure that the domain <app-name>email.com (production) and <app-name>email.net (staging) is correctly setup in Mailgun to accept incoming emails. Example: For NeetoForm it would be neetoformemail.com (production) and neetoformemail.net (staging).

15. Add the following to ApplicationMailbox

routing /neetoemaildelivery/i => "neeto_email_delivery_engine/inbound"

16. Override skip_rate_limits? for NeetoEmailPipelineEngine::MailDeliveryJob to skip rate limiting if user has their SMTP integration configured.

# app/overrides/neeto_email_pipeline_engine/jobs/mail_delivery_job_override.rb
NeetoEmailPipelineEngine::MailDeliveryJob.class_eval do

  private

    def skip_rate_limits?
      is_smtp_email_job?
    end

    def is_smtp_email_job?
      return false if params[:delivery_account].blank?

      account_klass, account_id = params[:delivery_account].values_at(:klass, :id)
      delivery_account = account_klass.constantize.find_by(id: account_id)

      delivery_account.present? &&
        delivery_account.is_a?(NeetoEmailDeliveryEngine::SmtpAccount) &&
        delivery_account.active?
    end
end

Frontend package

Installation

1. Add the neeto-email-delivery-frontend package to the package.json

   yarn add @bigbinary/neeto-email-delivery-frontend

Instructions for development

Check the Frontend package development guide for step-by-step instructions to develop the frontend package.

Usage

Exported components

EmailDeliveryScreen

| Prop | Type | Description | | ----------------------- | ----------- | ------------------------------------------------------------ | | canManageIntegrations | boolean | Authorization/permission check for this feature. | | ownerId | uuid/string | ID of the owner record. | | indexRoute | string | Index route for the email delivery page. | | ownDomainSetupRoute | string | Nested route for the SparkPost "Own Domain" setup component. | | ownDomainVerifyRoute | string | Nested route for the SparkPost "Verify Domain" component. |

SparkpostDomainSetup

| Prop | Type | Description | | ----------------------- | ----------- | ---------------------------------------------------------------------------- | | canManageIntegrations | boolean | Authorization/permission check for this feature. | | ownerId | uuid/string | ID of the owner record. | | alreadyVerifiedRoute | string | Route for the "already verified" page; typically the email delivery index. | | verifyRoute | string | Route for the verification page that shows DKIM records. | | onCancelRoute | string | Route to redirect users on cancellation; typically the email delivery index. |

SparkpostDomainVerify

| Prop | Type | Description | | ------------------------- | ----------- | ------------------------------------------------ | | canManageIntegrations | boolean | Authorization/permission check for this feature. | | ownerId | uuid/string | ID of the owner record. | | emailDeliveryIndexRoute | string | Index route for the email delivery page. |

Business logic for sparkpost verification

• If the domain is not created on Sparkpost

  • Create the domain in sparkpost
  • Present the DKIM values to the user and ask the user to add it to their DNS
  • Present a verify button.
  • On verification, we make an API call to NeetoTower with domain and current organization.guid as parameters.

• If the domain is created on Sparkpost but not verified, then

  • Present the DKIM values to the user and ask the user to add it to their DNS
  • Present a verify button.
  • On verification, we make an API call to NeetoTower with domain and current organization.guid as parameters.

• If the domain is created on Sparkpost and verified, then (New flow)

  • Make a call to NeetoTower to check if the domain + current organization_guid record is present.
  • If the response from NeetoTower says the domain + current organization_guid exist, then just approve the domain automatically.
  • If the response from NeetoTower says the domain + current organization_guid doesn't exist, then
    • Show a custom TXT value and ask the user to add it to their DNS.
    • Present a verify button which checks the above TXT value.
    • On verification, we make an API call to NeetoTower with domain and current organization.guid as parameters.

Sparkpost Domain statuses

• pending - The default status when the sparkpost domain is created.
• sp_dkim_verified - The domain's DKIM record has been verified in Sparkpost.
• sp_dkim_unverified - The domain's DKIM record is yet is not verified in Sparkpost.
• failed - Failure state when any of the Sparkpost API's for verification fail.
• txt_unverified - This is for the case when a domain has been verified for a different organization. We store the domain+organization mapping in NeetoTower. If the domain+organization mapping is not present in NeetoTower but the domain is verified on Sparkpost, then we use this state to indicate that the domain needs further verification via validating TXT records. If the domain+organization mapping is already present in NeetoTower, then we directly mark the domain as active.
• active - The final state. If active, then we will allow sending emails via this domain.

Outlook integration flow

Ensure that MAILGUN_INGRESS_SIGNING_KEY environment variable is set in the host application. This is needed for the verification of the Outlook account.

1. Connect Outlook from UI

   • User clicks the "Connect Outlook" action in the host application.
   • The engine redirects the user to Microsoft OAuth using User.Read, Mail.Send, and offline_access scopes so that we can send emails and refresh tokens without re‑auth.

2. Store Outlook account and send verification email

   • After a successful OAuth callback, NeetoEmailDeliveryEngine::Integrations::Outlook::ConnectService creates or updates an OutlookAccount and sets its status to inactive.
   • The service generates a test_email_token, persists it in the account metadata (along with timestamp and subject), and sends a test email via NeetoEmailDeliveryEngine::Outlook::SendEmailService using the Microsoft Graph sendMail API to a generated verification address like outlook.neetoemaildelivery@<application_name>email.(com|net).

3. Inbound verification handling

   • Incoming emails to the verification address are processed by NeetoEmailDeliveryEngine::InboundMailbox.
   • The mailbox identifies Outlook verification emails by recipient and subject, extracts the test_email_token from the subject, and finds the corresponding OutlookAccount.
   • If the email is from a legitimate user mailbox and the account is not already active, the mailbox updates the account status to active. If the sender looks like an Outlook shadow account (e.g. [email protected]), the mailbox flags the account as shadow_account and marks it as forbidden instead of activating it.

Disconnection emails

In case the email delivery fails using the Gmail/Outlook integrations, we disconnect the integration and send out an email. To customize the email, the host application needs to add an override so that they are in control of the mailers.

Override the base_email_service.rb as mentioned above in installation step 9.

Automation testing endpoints

The engine exposes testing endpoints under /neeto_email_delivery/api/v1/testing for use in automation tests. They let tests connect Gmail/Outlook and verify sent emails without the browser OAuth flow, and are disabled in production. Refer to the Testing Endpoints Guide for the request/response shapes and instructions for obtaining refresh tokens.

## Instructions for Publishing

Consult the
[building and releasing packages](https://neeto-engineering.neetokb.com/articles/building-and-releasing-packages)
guide for details on how to publish..