@serve.zone/coremail

CoreMail is the authenticated mail persistence and delivery-orchestration service for serve.zone workloads. dcrouter serves the CoreMail mail gateway it connects to, and its control client — Cloudly's CoreMail manager, or Onebox's CoreMail provider — delivers its control bootstrap and desired state and reads its reconciliation status. From 34.0.0 CoreMail runs only on a runtime that injects its workload replica identity (see Configuration).

Installation

pnpm add @serve.zone/coremail

CoreMail installs on Linux only. Its SMTP submission listener is built on @push.rocks/smartmta, which declares os: linux and ships the prebuilt Rust mailer-bin for linux-x64 and linux-arm64 — CoreMail's own runtime target. Installing on macOS or Windows fails at the dependency, by design; develop against a Linux container or host.

The package exports the service lifecycle and its strict configuration reader:

import { CoreMail, readCoreMailConfig } from '@serve.zone/coremail';

const coreMail = new CoreMail(readCoreMailConfig());
await coreMail.start();

process.once('SIGTERM', () => {
  void coreMail.stop();
});

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Architecture

The integration architecture is:

workload
  └─ @serve.zone/platformclient
       └─ authenticated TypedSocket + grant-authorized HTTPS transfers
             └─ CoreMail
                  ├─ NoSQLDB: bindings, grants, submissions, delivery state
                  ├─ SmartBucket: message parts and immutable MIME objects
                  └─ authenticated TypedSocket ── dcrouter mail gateway

dcrouter ── grant-authorized HTTPS transfers ── CoreMail transfer origin

Cloudly CoreMail manager / Onebox CoreMail provider
  └─ authenticated control TypedSocket ── CoreMail: desired state, reconciliation status

CoreMail enforces these boundaries:

• A workload session's authority is server-owned: binding, tenant, service, binding state, capabilities, the credential id and version that authenticated, and the transfer origin. Every workload request rebuilds it from the current desired state and compares it with the session's; any difference, or a credential no longer accepted, closes the socket with 4403. Neither the desired-state epoch nor the binding revision is part of it, so an apply that changes only the epoch or other bindings, or that rotates the binding's credential while the session's is still accepted, leaves the session in place.
• A refused workload authentication is answered as AUTHENTICATION_FAILED and leaves the socket open. A refusal by the per-identity Argon2id limiter is retryable: true, with retryAfterMs until the identity's next token when its budget is spent; a wrong, unknown or expired credential is not retryable.
• Active and draining bindings derive their exact permitted operation set from the shared contract; disabled bindings cannot authenticate.
• Every reconnect requires a new Argon2id-authenticated handshake.
• No CoreMail session negotiates a contract major: interfaces 32.0.0 puts the protocol offer on the registration carriers and gives the CoreMail contracts none, so the control, workload and gateway sessions are first-contact exchanges judged by their exact contract alone.
• TypedSocket tags are routing metadata, never authentication.
• Message bytes never travel as JSON, base64 RPC payloads, or VirtualStreams. One-time grants authorize bounded HTTP PUT or GET transfers.
• Only grant-token SHA-256 digests are persisted. Plaintext bearer and credential material is never stored or logged.
• @lossless.org/client/nosqldb owns all structured persistence, indexes, cursor pagination, fixed-window quotas, pending-inbound gauges, compare-and-set transitions, and worker leases.
• SmartBucket owns all message parts and immutable serialized MIME objects, with exact length and SHA-256 reconciliation.
• The service has no filesystem persistence path and no raw MongoDB access.

Runtime surfaces

One strict hostname surface is exposed:

• GET /live reports process liveness.
• GET /ready reports database, exact object-storage, desired-state, authenticated gateway, and SMTP submission listener readiness.
• GET|PUT /transfers/:grantId consumes one-time transfer capabilities.
• GET /socket upgrades to the authenticated TypedSocket RPC transport.

When the desired state enables it, an SMTP submission listener (MSA) runs on its own port. It offers STARTTLS and accepts AUTH only on an encrypted session — the username is a bindingId and the password any accepted credential secret of that binding. Accepted messages go straight to the durable outbound pipeline as exact bytes; the embedded SMTP stack stores, queues and forwards nothing. A listener whose certificate or private key material does not resolve stays down and is reported unavailable.

HTTP TypedRequest and built-in routes are disabled. WebSocket messages are limited to 64 KiB, unauthenticated connections have five seconds to complete their first authentication request, and HTTP connection/header/request deadlines are enforced by SmartServe.

Configuration

CoreMail reads its configuration from the process environment, as @serve.zone/interfaces states it ("CoreMail runtime environment"):

• its replica identity: SERVEZONE_WORKLOAD_SERVICE_ID, SERVEZONE_WORKLOAD_REPLICA_ID, SERVEZONE_WORKLOAD_ATTEMPT_ID and SERVEZONE_WORKLOAD_IMAGE_DIGEST (data.workloadReplicaIdentityEnvironmentNames). CoreMail runs only on a runtime that injects them: Pallet and Onebox from their releases that consume @serve.zone/interfaces 32.62.0 and inject the identity; on an earlier runtime it does not start. CoreMail refuses to start naming every one that is missing or invalid, never its value;
• its database and object storage under the names its platform bindings deliver (data.coreMailStorageEnvironmentNames): MONGODB_URL and MONGODB_DATABASE from the Corestore database binding, S3_ENDPOINT, S3_REGION and S3_BUCKET from the object-storage binding, and that binding's S3_ACCESS_KEY and S3_SECRET_KEY secrets;
• its own data.coreMailRuntimeKeys: COREMAIL_CONTROL_BOOTSTRAP, and optionally COREMAIL_PORT, COREMAIL_HOSTNAMES and COREMAIL_MONGODB_MAX_POOL_SIZE;
• the secrets its desired state references by name.

Secret material is delivered by launching the workload through workloadinit. The platform integration must write the verified map and secret files; workloadinit injects their exact values into the process environment immediately before executing CoreMail. CoreMail does not fetch, decrypt, or persist runtime secret material itself. Malformed startup configuration terminates startup. Missing desired-state secret references keep the affected gateway or cursor operation unavailable.

The SMTP submission listener is configured entirely through desired state (smtp): enabled, port, hostname, and tls.certificatePemSecretKey / tls.privateKeyPemSecretKey, which name runtime secret keys holding PEM text — never the PEM values themselves. An omitted or disabled smtp section keeps the API-only surface; an enabled one whose secret keys do not resolve leaves the listener down. Changing any of these values restarts the listener in place.

/opt/serve.zone/runtime-assets/workloadinit/workloadinit run \
  --map /run/serve.zone/workloadinit-map.json -- node cli.js

| Variable | Purpose | | --- | --- | | SERVEZONE_WORKLOAD_SERVICE_ID | Runtime-injected service identity; the control bootstrap must name the same service | | SERVEZONE_WORKLOAD_REPLICA_ID | Runtime-injected logical replica | | SERVEZONE_WORKLOAD_ATTEMPT_ID | Runtime-injected execution attempt; reported to the control client, and its digest is the outbound worker's lease owner | | SERVEZONE_WORKLOAD_IMAGE_DIGEST | Runtime-injected sha256: image digest | | MONGODB_URL | Database binding URL consumed only by the document store | | MONGODB_DATABASE | Database binding database name | | S3_ENDPOINT | Object-storage binding endpoint: a canonical http:/https: origin, whose scheme and port the connection uses | | S3_REGION | Object-storage binding signing region | | S3_BUCKET | Object-storage binding bucket, a canonical S3 bucket name | | S3_ACCESS_KEY / S3_SECRET_KEY | Object-storage binding credential secrets | | COREMAIL_PORT | Listener port; defaults to 3000 | | COREMAIL_HOSTNAMES | Comma-separated canonical strict-surface hostnames; defaults to coremail.serve.zone | | COREMAIL_MONGODB_MAX_POOL_SIZE | Bounded document store connection pool; defaults to 50 | | COREMAIL_CONTROL_BOOTSTRAP | Verifier-only control bootstrap JSON | | ${gateway.credentialSecretKey} | Runtime-only dcrouter peer credential referenced by desired state | | ${cursorKeys[].secretKey} | Runtime-only HMAC keys referenced by current/retiring desired-state cursor keys |

COREMAIL_CONTROL_BOOTSTRAP contains verifier material only. The matching plaintext control credential is held only by the control client. Desired state contains key references for gateway and rotating cursor credentials, never their plaintext values. gateway.endpointUrl is the dcrouter TypedSocket endpoint. gateway.coreMailTransferOrigin is CoreMail's canonical HTTPS origin; dcrouter must return that exact origin during authentication before it can use CoreMail-issued path-only transfer grants.

The coreMailAuthenticateControl response and every reconciliation status carry the reporting replica as data.IWorkloadReplicaIdentity (serviceId, replicaId, attemptId, imageDigest), exactly as its runtime injected it. CoreMail before 34 reported the deprecated data.ICoreMailReplicaIdentity; a control client reads either through data.normalizeCoreMailReplicaIdentity and must be on @serve.zone/interfaces 32.62.0 or later before CoreMail 34 runs.

CoreMail keeps its gateway session through a gateway outage of any length. Each socket is one connection that authenticates before it counts as connected; when one ends, or cannot be opened or authenticated, CoreMail opens the next after a backoff that doubles from one second to one minute, and warns about an outage at most once a minute. Only a socket that stayed authenticated for a minute resets the backoff when it ends, so a gateway that accepts the credential and drops the socket at once is one outage, not a login every second. Only a definitive refusal of the credential (AUTHENTICATION_FAILED with retryable: false) stops it: CoreMail logs it at error, reports the gateway not ready, and opens no socket with that credential again until the desired state names another gateway credential. Every TypedSocket peer — the gateway, the control client and every @serve.zone/platformclient workload — must speak TypedSocket 9: the exact package-major handshake refuses a TypedSocket 8 peer before any request.

Persistence and delivery behavior

Submissions, inbound deliveries, quota counters, idempotency keys and inbound cursors belong to the binding identity: tenant, service and binding. The binding revision scopes none of them and is only echoed in the reconciliation status, so a credential rotation changes nothing a workload has stored, used up or paged through.

Outbound submission preparation is idempotent per authenticated binding. The first durable insert consumes one atomic fixed-UTC minute and day quota unit; exact replays consume none. Every part has a unique immutable object key and exact declared integrity. Finalization reads verified parts, produces deterministic MIME with no Bcc header, publishes that MIME exactly once, and queues a lease-fenced worker. The downstream transport identity is the CoreMail submission ID and remains stable across ambiguous responses and retries.

Every outbound submission records how it entered CoreMail. API submissions are composed from declared parts as described above. An SMTP submission is accepted as exact RFC822 bytes: it has no part descriptors, its bytes become its immutable MIME unchanged, and it enters the same lease-fenced worker in the same state a finalized API submission does. It is subject to the same binding authority, sender authorization, recipient validation, serialized-MIME byte budget, and minute and day quota. Records written before this discriminator existed are read as API submissions.

Inbound SMTP recipient resolution returns short-lived opaque handles, each bound to a digest over the canonical JSON (keys sorted) of the envelope and connection source, so completion recomputes the same digest from the stored handoff whatever key order storage returns. The gateway uploads one immutable MIME object, after which CoreMail resumes a transactional saga that consumes handles and creates one delivery per binding. Only active, inbound-capable bindings claim their configured recipients. Draining, disabled, and otherwise unowned recipients return unhandled, which allows the gateway to continue its next configured resolver instead of issuing an SMTP rejection. Pending, fetching, and fetched deliveries consume the binding's pending-inbound quota until the first successful acknowledgement. Workloads list deliveries through a bounded cursor, fetch MIME through a one-time grant, and may acknowledge only after the exact fetch completes. Cursor signatures rotate through current and bounded retiring desired-state keys without exposing runtime secret material.

Every gateway-facing handler answers with the same privacy-safe typed error envelope the workload handlers use, so the transport can tell a retryable QUOTA_EXCEEDED or STATE_CONFLICT from a permanent refusal without ever receiving a cause string.

Inbound deliveries carry the Message-ID and Subject read from a bounded prefix of the stored message. The gateway message descriptor cannot carry either, and a header scan that fails never costs a delivery — the fields simply stay absent.

Per-binding counters are kept for each UTC day: submittedApi and submittedSmtp when a submission is published to the worker, delivered, deferred, failed and deadLettered on outbound transitions, and received, acknowledgedProcessed and acknowledgedDiscarded on inbound ones. Each increment commits in the same transaction as the durable transition it counts, so a replayed transition — a repeated finalization, a re-applied gateway status, a replayed handoff completion or acknowledgement — counts nothing. Counter rows are retained for 90 days.

Desired-state activation uses immutable snapshots and a strictly increasing configEpoch compare-and-set pointer. Replayed or stale configurations cannot replace newer authority.

Terminal outbound submissions and acknowledged inbound deliveries retain their records and owned objects for 30 days. Abandoned outbound preparations and handoff/idempotency receipts also expire after 30 days. Expired transfer grants and recipient handles retain their records for another 24 hours. Pending, fetching, and fetched inbound deliveries are never age-purged; their MIME stays stored until acknowledgement and the acknowledged-delivery retention period have elapsed.

Development

pnpm install
pnpm run build
pnpm test
pnpm exec tsrepo check 'test/**/*'

The production image is built natively for linux/amd64 and linux/arm64:

pnpm run build:docker
pnpm run release:docker

It is built from Dockerfile_##version##, so every build publishes the release version tag and nothing else. coremail:latest is never moved again — it keeps the last 1.x build it held — and consumers pin the version tag.

A release runs the full test suite and a build before it publishes anything, and it qualifies the built image before it is pushed: tsdocker runs test/test_latest.sh inside the image on every target architecture. That script asserts the image contents and the versions it carries, that node cli.js refuses naming every missing SERVEZONE_WORKLOAD_* variable, and that the CoreMail HTTP surface binds COREMAIL_PORT and answers /live and /ready the way the HEALTHCHECK dials them. The image ships neither a database nor object storage, so /ready answers 503 there with the readiness document for "listening, external dependencies absent".

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the license.md file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

Company Information

Task Venture Capital GmbH Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at [email protected].

By using this repository, you acknowledge that you have read and understood these terms. If you do not agree with them, you must not use the repository.