@volter/twin-smtp
v0.1.38
Published
Local SMTP twin (RFC 5321 submission server: EHLO/AUTH/MAIL/RCPT/DATA over raw TCP, messages folded into the twin ledger, twin-only HTTP inspect sidecar) built on @volter/world-core.
Readme
@volter/twin-smtp
A local SMTP submission server your app's unmodified mail client talks to over raw TCP.
Point SMTP_HOST/SMTP_PORT (or Cal.com's EMAIL_SERVER_HOST/EMAIL_SERVER_PORT, or NextAuth's
EMAIL_SERVER) at it and an unmodified nodemailer sends real SMTP to it: greeting, EHLO,
AUTH PLAIN/LOGIN, MAIL FROM/RCPT TO/DATA with dot-stuffing, faithful reply and enhanced
status codes. Every accepted message is parsed and folded into the twin ledger, and read back over
a twin-only HTTP inspect sidecar — so an agent can pull the login code out of the mail the app just
sent, without a real mailbox anywhere.
Nothing is ever put on the wire. The twin is a sink, and smtp.store.never_puts_mail_on_the_wire
asserts it.
world-smtp serve --port 1025 --inspect-port 8025 --root .volter/world/smtp
# app: SMTP_HOST=127.0.0.1 SMTP_PORT=1025
curl -s localhost:8025/twin/messages/latest | jq -r '.text'CLI
| command | what it does |
|---|---|
| world-smtp serve [--port N] [--inspect-port N] [--root DIR] [--hostname NAME] [--read-only] | bind the SMTP listener + the inspect sidecar, which also serves MailHog's API (both loopback-only) |
| world-smtp conformance [--root DIR] | probe every RFC 5321 command and the sidecar; non-zero on any failure |
There is no mirror command — see No UI mirror below.
Class doctrine — read this before copying the pack
This is the catalog's first raw-protocol (non-HTTP) pack, and five things work differently here than in an HTTP twin. Each was a decision, and each has a reason.
1. Interception is app-read ENV, because there is no host to rewrite
Every other pack is intercepted by packages/world-core/inject.cjs's VENDOR_HOSTS: the
injector patches http/fetch, sees api.stripe.com, and redirects it. Neither half of that
works for SMTP. The traffic is a line protocol on a raw TCP socket, so the injector never sees it;
and there is no vendor host to match on anyway — the relay is whatever the operator configured
(smtp.gmail.com, email-smtp.us-east-1.amazonaws.com, localhost:1025).
So the interception seam is the env var the client is configured with, which for SMTP is not a hint about a touchpoint — it is the touchpoint. The world injects all three shapes, because which one an app reads is not knowable from outside:
| env | who reads it |
|---|---|
| SMTP_HOST / SMTP_PORT | nodemailer's own transport config; the classic pair almost every framework reads |
| EMAIL_SERVER_HOST / EMAIL_SERVER_PORT | Cal.com's split form |
| EMAIL_SERVER (smtp://host:port) | NextAuth's Email provider, which takes a connection URL |
Wired at APP_READ_ENDPOINT_ENV in world-runtime/src/init.ts, as injectEnvTemplates over
${host}/${port} — the same mechanism livekit uses for its ws:// media plane, one layer
further down. scripts/vendor-hosts.test.ts carries the matching its descriptor hostsNone ruling reason, and
volter-world covers now reports the raw-mail signal (SMTP_HOST, EMAIL_SERVER_HOST,
MAILGUN_SMTP_HOST, a nodemailer dependency) as a detected vendor rather than the
unknown-sdk row it produced before this pack existed.
Two ports, deliberately, and the second one is derivable. SMTP is server-speaks-first: the server must emit its 220 banner before the client says anything. Multiplexing HTTP onto the same socket therefore means delaying that banner while peeking for an HTTP verb — a race against every conforming client, to save one port. The twin binds the protocol port (the one the app's env points at) and the inspect port separately, and prints both.
The world runtime models one port per service, so it allocates the SMTP port only. An omitted
--inspect-port binds an ephemeral port, reported as server.inspectPort and printed by the CLI;
a World that reads the mail back pins --inspect-port (the dub-stress harness pins 8025, MailHog's
port). The sidecar never derives a port nobody handed it: port + 1 took the port the World had
allocated to the next service, which then failed with EADDRINUSE and brought the colocated host
down, deterministically in the tab's engine, whose allocator hands ports out in sequence.
Loopback bind only, and there is no option to widen it. This server fakes auth locally and
accepts any sender: on a wildcard bind that is an unauthenticated open mail relay listening on every
interface of the operator's machine. smtp.wire.loopback_bind_only proves the bind by connecting
to this host's own routable addresses, not by reading the constant back.
2. The seam is a REDUCER, not a request handler
scripts/mutation-test.ts sabotages a pack by swapping a module export. A twin whose only seam is a
listening socket is therefore untestable by this repo's gates — a verify that dials a real socket
booted elsewhere bypasses every saboteur and survives having proven nothing.
So the protocol lives in smtp-twin.ts as a reducer over an explicit session value —
openSmtpSession(config) → {session, reply} and handleSmtpCommand({session, line, …}) →
{session, reply, close?} — and smtp-server.ts (Bun.listen) is a thin adapter owning only
framing. The wire path and the verify path run the same code.
Two details that are load-bearing rather than stylistic:
- The session driver lives in a separate module (
smtp-session.ts). A caller in the same module resolves its local binding, which a module mock cannot reach — a driver next to the reducer would have made every verify a survivor. - The saboteur value is protocol-shaped.
httpEmpty's{status, body}is meaningless here, so thePACKSentry usessmtpEmpty(a contentless250and a stateless session) andsmtpGreetEmpty(a contentless 220). Both are blank successes, so a verify that merely asked "did it answer 250?" would survive — teeth have to come from the reply text, the enhanced status code, or the ledger. Current result: 87/87 API verifies regress under the dead engine.
The mutation gate is not the whole story, and this pack is the evidence. Two rounds of
adversarial review found ~34 real defects behind a green gate — a stricter-than-the-RFC address
parser that rejected root@localhost (the twin's own headline use case), a Received: header in a
form no MTA emits with a verify that pinned the wrong format, BODY=BINARYMIME answering 250 for an
extension filed as a todo, an AUTH loop that could never be disconnected, an RFC 2047 bug that
silently dropped characters out of exactly the OTP this pack exists to surface, and a connector that
folded a failed probe over real state. None of it was reachable by any gate in this repo. Every fix
is pinned by a revert-matrix cell that reddens by name.
3. State lands in the ledger; the read-back is a twin-only sidecar
Accepted messages are message.send actions on a message subject — deliberately the same
operation name postmark's twin uses, because it is the same event. Two ledger decisions:
- A fresh queue id per transaction is what makes a message an occurrence.
applyTwinWritededupes by (content +occurredAtmillisecond), which is right for an idempotent resource write and wrong here: a real MTA handed the same bytes twice queues two, and collapsing them would answer 250 with a queue id holding nothing. The ordinal minted from the action log sits inside the subject id, so two identical deliveries are two subjects (smtp.store.identical_messages_both_land). This pack originally also passed the kernel'suniquenessseam to say so — and the adversarial review proved that parameter was decorative here (the ordinal was already in the hashed content), so it was removed rather than left as a comment defending a mechanism nothing could test. - Queue ids ratchet off the ACTION LOG, not the projection. A purged message vanishes from the
projection but keeps its ordinal in the log, so a projection-derived mint would re-issue a retired
queue id (
smtp.store.queue_id_survives_purgeis the dirty-state pin).
Why the inspect surface is a sidecar, and why it is NOT in the manifest. SMTP is write-only: a
submission server has no verb that hands a message back. ../../../docs/contributing/adding-a-twin.md §6 already settles
the mirror image — a read-only vendor gets "twin-only routes outside the vendor's surface… clearly
namespaced and out of the capability manifest… counting them as coverage would be padding." SMTP is
that rule reflected, so the same answer applies: everything twin-only lives under /twin/, none of it appears
in the capability manifest, and it is gated by smtp-inspect.test.ts plus the end-to-end probes in
smtp-conformance.ts instead. MailHog's /api/* routes on the same port are a vendor's surface
and are counted (below).
Its list/detail/paging structure parallels postmark's /messages/outbound +
/messages/outbound/<id>/details split. It does not copy Postmark's PascalCase field naming:
that casing is a fidelity claim about Postmark's API, and wearing it on a surface no vendor
publishes would imply this twin is imitating something. It is not.
| route | |
|---|---|
| GET /twin/messages?to=&from=&subject=&search=&count=&offset= | newest-first summaries; search scans decoded bodies — the "find the OTP" query |
| GET /twin/messages/<queueId> · GET /twin/messages/latest | the full record: headers, decoded text/html, attachments, raw bytes |
| GET /twin/mail?to=<address> | the recipient's mailbox: every message whose envelope names that address (whole, case-insensitive), newest first, with its bodies, its Reply-To and its attachments (name, type, size, download path, and a calendar invite's event); a browser (Accept: text/html) gets a readable page, as the Resend twin's /_twin/mail inbox draws it — the stand-in for the recipient's mail client, which is outside SMTP |
| GET /twin/messages/<queueId>/attachments/<n> | the n-th attachment's decoded bytes (0-based), under its own type and a Content-Disposition: attachment naming the file |
| DELETE /twin/messages | purge (soft-delete; the queue-id ratchet survives it) |
| GET /twin/health | {ok, protocol, service, messages} |
Anything else on that port 404s loudly — answering a stray /v1/… with a 200 would be the
invented-surface false-green.
MailHog's HTTP API on the same port
Apps whose CI stands up MailHog beside their SMTP env read mail
back through MailHog's API at :8025 (Dub's Playwright specs poll
GET /api/v2/search?kind=to&query=<addr>). The inspect sidecar answers those routes too, over the
same stored messages, so world-smtp serve --inspect-port 8025 stands where MailHog stood. Unlike
/twin/*, these are a real vendor's published surface, so they are in the manifest as area
mailhog (src/smtp-mailhog.ts).
| route | answer |
|---|---|
| GET /api/v2/messages?start=&limit= | {total,count,start,items}, newest first; limit defaults to 50, capped at 250 |
| GET /api/v2/search?kind=from\|to\|containing&query=&start=&limit= | the same envelope over matching messages; another kind or an empty query is 400 |
| GET /api/v1/messages | every message (up to 1000) newest first, as a bare array |
| GET /api/v1/messages/{id} | one message; an unknown id is 200 null, as MailHog answers |
| DELETE /api/v1/messages · DELETE /api/v1/messages/{id} | purge all / one through the same soft-delete purge as DELETE /twin/messages; an unknown id is 500 |
Each item is MailHog's data.Message: ID, From and To paths (Relays, Mailbox,
Domain, Params), Content (Headers as arrays, the raw Body after the header block,
Size in bytes, MIME, which MailHog leaves null at the top level), Created, MIME (the parts
of a multipart/ message), and Raw (From, To, Data, Helo). Paths match exactly, every
MailHog route answers OPTIONS with an empty 200 (including the ones left todo), and a search page starting past its hits reports
total: 0, as MailHog's does.
Grounded in MailHog's source (master):
| behavior | source |
|---|---|
| route table | MailHog-Server api/v1.go:47-65, api/v2.go:34-49 |
| v2 envelope, start/limit defaults and cap | api/v2.go:72-97; messages api/v2.go:99-121; search and its 400s api/v2.go:123-154 |
| v1 list, load (null for unknown), delete all, delete one | api/v1.go:122-142, api/v1.go:144-166, api/v1.go:239-254, api/v1.go:344-359 |
| newest-first paging, case-insensitive search over path then To/From header, containing over body then headers; unknown load and delete | mailhog/storage memory.go:41-160, memory.go:163-172, memory.go:194-199 |
| the Message, Path, Content, SMTPMessage, MIMEBody JSON | mailhog/data message.go:50-87 |
| header parsing (last value of a repeated field wins, continuation lines appended), Size = length of DATA | message.go:284-320 |
| MIME parts only for multipart/, split on --boundary | message.go:215-255 |
| added Message-ID, appended Received, Return-Path: <from> | message.go:111-145 |
Where the twin differs, on purpose: the ID is <queueId>@<server hostname> rather than MailHog's
random base64 id (message.go:29-43), so reads are deterministic and the queue id stays visible;
the appended Received value is the twin's own RFC 5321 trace line; a page starting exactly at
total is empty, where MailHog's arithmetic returns its oldest row; the multipart boundary is read
by a regex rather than Go's mime.ParseMediaType, so a malformed Content-Type may still split. The event stream, websocket,
download, release and Jim routes are todo in the manifest.
4. What the twin does NOT have is as much of the surface as what it does
The inverse false-green — serving surface the vendor doesn't have — has no oracle and no gate, so it is the class this pack has to police by reading. Three places where the answer is a refusal:
- DSN parameters are refused
555, not accepted-and-ignored. This server never advertisesDSN, and RFC 5321 §4.1.1.11 makes an unadvertised parameter an error. - Parameter VALUES are validated, not just keywords.
BODY=BINARYMIMEis refused501: BINARYMIME is RFC 3030, which this twin files as atodo, and answering 250 would be the twin saying yes to an extension it does not have. (An allowlist of keys alone let exactly that through, and it also meantSIZE=notanumbersilently bypassed the whole declared-size gate.) SMTPUTF8changes behaviour or it is decoration. RFC 6531 §3.4 is a rule in both directions, so a non-ASCII mailbox without the parameter is553 5.6.7. An advertised keyword that gated nothing would be the same disease as accepting one that was never advertised.
5. STARTTLS is deliberately NOT advertised
RFC 3207 makes the extension optional, and the thing this twin stands in for is a plaintext relay on
loopback, which does not offer it either. Advertising it would be actively worse than useless:
nodemailer upgrades opportunistically the moment a server advertises STARTTLS, so a twin without a
real, trusted TLS layer would break every unmodified client on the machine — recoverable only by
telling the operator to set rejectUnauthorized: false, i.e. to modify the client, which the
fidelity bar forbids. STARTTLS therefore gets RFC 3207's own 502 from a server without the
extension, and negotiated TLS is a filed todo (smtp.ext.starttls_negotiation).
The other half is proven, not asserted: because the twin is honest about not offering it, a
requireTLS: true nodemailer correctly refuses to send in the clear
(smtp-sdk.integration.test.ts).
Coverage
Partial and honest. The capability manifest (src/smtp-capabilities.ts) is the real vendor
surface as the denominator — and for a protocol twin the "vendor" is a set of RFCs, enumerated
top-down from their own section structure rather than from anything this twin implements:
- RFC 5321 (SMTP) — §4.1.1's complete command list, §4.1.2's path grammar (source routes, the
null reverse-path), §4.2's reply-code theory, §4.4's
Received:trace header, §4.5.2's dot-stuffing transparency rule, §4.5.3.1's size limits. - RFC 5322 + RFC 2045/2046/2047 — header folding, MIME structure and transfer encodings, encoded-words.
- One
extensionsentry per ESMTP RFC whether or not this twin implements it: RFC 1870 SIZE, RFC 2034 ENHANCEDSTATUSCODES, RFC 2920 PIPELINING, RFC 3030 CHUNKING/BINARYMIME, RFC 3207 STARTTLS, RFC 3461 DSN, RFC 4954 AUTH (+ RFC 4616 PLAIN), RFC 6152 8BITMIME, RFC 6531 SMTPUTF8, RFC 8314 implicit TLS.
See the generated table in the repo README for the live
numbers. Read the core column with the RFC in mind: it is high because RFC 5321's own core is
small and closed, and this pack implements nearly all of it — not because the tiering is generous.
The core gaps that remain are named rather than buried: negotiated STARTTLS
(smtp.ext.starttls_negotiation) and implicit TLS on 465 (smtp.wire.implicit_tls_465), which
together are how most production SMTP integrations actually connect. Everything else outstanding is
an extension RFC, and all of it is in the denominator.
The adversarial review also moved three entries down from core to common
(readonly_refuses_data, loopback_bind_only, never_puts_mail_on_the_wire): they are real and
proven, but they describe the twin's posture rather than surface an SMTP client exercises in week
one, and counting twin-side rungs as vendor core is how a coverage number stops meaning anything.
Fidelity is proven against the real nodemailer 7.0.13, driven unmodified over a real loopback
TCP socket (src/smtp-sdk.integration.test.ts): its own verify() handshake, AUTH PLAIN, its
dot-stuffing (both halves real implementations that have never met), a pooled multi-transaction
connection, and its handling of the twin's 550/552/554 refusals. "Unmodified" means un-patched, not
un-configured — pointing a transport at host/port is what a real integrator does, and it is the
entire wiring story for this pack.
That test earned its keep immediately: nodemailer's default shape for text + html + an attachment is
multipart/mixedwrappingmultipart/alternative, and a single-level MIME walk projected an empty body — silently losing exactly the login code this twin exists to surface, while every reply code stayed green. The parser is recursive (depth-capped) because of it.
Fidelity quirks deliberately preserved
- Enhanced status codes are suppressed for a HELO-only session. RFC 2034 §4 permits them only
once the client has issued EHLO and the server advertised
ENHANCEDSTATUSCODES; an old client is entitled to classic three-digit replies. (This is why the over-long-line refusal lives in the engine rather than the socket adapter — the adapter cannot know whether the session negotiated.) - A single-label domain is accepted.
Domain = sub-domain *("." sub-domain):root@localhostandapp@webare legal, and they are the dev-mail address shapes this twin exists to serve. 552for an oversized payload is deferred to the terminating dot, per RFC 1870 §6.3 — the bytes are consumed first, and nothing is stored.VRFYanswers252 2.0.0, RFC 5321 §3.5.3's "will not verify". Not2.1.5: RFC 3463's X.1.5 means "destination address valid", which is exactly the claim a 252 declines to make.RCPT TO:<>is501. The null path is legal as a reverse path only.- Obsolete source routes parse and are ignored (
<@relay:user@host>→user@host). - The
Received:trace header is RFC 5321 §4.4 ABNF: a bracketed address-literal (([127.0.0.1])), an RFC 5322date-timewith a numeric+0000zone rather than the obsoleteGMT, and RFC 3848'sESMTPAwhen the session authenticated. AUTHis refused503inside a mail transaction (RFC 4954 §4), and AUTH failures count toward the hard-error drop — a server that fakes auth is the worst place to allow an unbounded credential loop.
No UI mirror
The API is the product, in the strongest form the rule admits. SMTP is a wire protocol with no
vendor, no console and no screen: when someone does its core job — an app sends transactional mail —
they write code, and there is no browser step at any point. So this pack ships no client/
directory and declares no ui-dimension capabilities at all, and ui-scope.json records
needsUi: false.
The tempting wrong answer is worth naming, because it looks plausible: Mailpit and MailHog do ship
web inboxes, so why not declare one as this twin's ui? Because that inbox is a view over
captured mail — over this twin's own scaffolding — not over any vendor surface. Reading mail back
is served here by the /twin/* inspect sidecar, which is excluded from the manifest for exactly the
same reason a seed route is. Its /twin/mail page (a person in a World reading the code an app mailed
them, as the Resend twin's /_twin/mail inbox serves one) is that same scaffolding drawn for a
browser: it is never declared as a capability, and it mirrors no vendor's screen.
What this twin does not do
- Node. The mail listener is raw TCP (
Bun.listen) and the connector dials withBun.connect; the kernel's HTTP server seam carries HTTP only, so this twin's wire runs under Bun (its inspect door, an HTTP server, is on the seam). Anode:netbranch is a ruled lane, not a gap in the protocol. - The twin has no MTA and never puts real mail on the wire: every accepted message is stored in the ledger and read back over the inspect sidecar. A submission twin is the final hop by construction, so there is no MX lookup or next-hop routing, and no downstream MTA to report a bounce (a scripted bounce is a filed todo). Spam filtering, greylisting and reputation are a real receiver's policy, and a simulated verdict would have no oracle behind it.
- A loopback twin holds no TLS certificate a client's trust store accepts, and the only
workaround modifies the client — see the STARTTLS decision in
smtp-twin.ts. SPF / DKIM / DMARC are verdicts about live DNS for the sending domain: the raw headers are stored, never evaluated.
Known gaps (filed, not hidden)
STARTTLS negotiation, CHUNKING/BDAT, DSN, implicit TLS on 465, CRAM-MD5/XOAUTH2, charset
conversion for non-UTF-8 bodies, message/rfc822 parts, a scripted bounce lifecycle, and relaying
accepted mail onward to a real MTA (smtp.connector.push_relay — pushPendingSmtpActions refuses
by name rather than draining silently, because real onward delivery is irreversible).
Connector
pull is shaped unlike every other pack's, because SMTP has no read path. What a real relay does
expose on every connection is its own identity and capability set — the 220 banner and the EHLO
keyword list — and that is what syncSmtpFromReal observes into a relay resource, so a twin
standing in for smtp.sendgrid.net can advertise what that relay advertises. It is idempotent, its
default occurredAt moves (so a relay capability that reverts across polls is not swallowed as
a phantom delta), and a refused probe throws in both shapes — a hard connection error and the
softer case where bytes came back but they are a 4xx greeting — rather than folding an empty relay
over real observed state.
liveSmtpProbe is the pack's single live call site and its only Bun.connect, guarded by a
fail-closed rate budget. The budget is declared even though scripts/rate-budget-coverage.test.ts
cannot see a raw socket at all: the scanner's blind spot is not permission. Its ceiling claims no
vendor fact — SMTP publishes no scalar limit and the real ceiling belongs to whichever relay the
operator names (Gmail meters recipients/day, SES a messages/second rate, Postmark concurrent
connections) — so it sits strictly under the kernel fallback at 10 relay connections per minute.
