@kilnjs/core
v0.1.3
Published
An opinionated application core for Fastify and Prisma: discovered routes, a compiled permission vocabulary, and a cache the database invalidates for you
Readme
@kilnjs/core
An opinionated application core for Fastify and Prisma. Routes are discovered rather than registered, the entry point is the framework's rather than the application's, and what it does beyond that is declared as modules in one file.
It knows nothing about any domain, and nothing about authorization. What exists, who may do what to it, and where a request is acting are declared by the application; the rules that decide are a module.
Install
yarn add @kilnjs/core
yarn add @prisma/client bullmq fastify ioredis zod # peersNothing else. The bundler, the TypeScript runner and the test runner are
core's dependencies, reached through the kiln command.
The toolchain
kiln dev run the application, restarting when what it imports changes
kiln build bundle the application into dist/
kiln start run the built application
kiln test run the test suite
kiln db the Prisma CLI, with the framework's schema and paths
kiln seed run the seeds in database/seeds, in path order
kiln routes print the route table discovery produced{
"scripts": {
"dev": "kiln dev",
"build": "kiln build",
"start": "kiln start",
"test": "kiln test"
}
}An application needs no esbuild.config.*, no jest.config.*, no
prisma.config.ts, no watcher config and no entry module. Each command
knows the conventions: the entry is generated, every file under src/ is a
build entry point because discovery imports by path at runtime,
dist/src/.application is written so discovery can find the output,
Prisma's CJS query compiler is aliased so its exports survive bundling, and
Jest is handed a generated config that maps each framework subpath to the
file its exports map declares.
That config also carries the application's own path aliases, read out of its
tsconfig.json so the compiler and the runner cannot disagree about where
@/ points. The file is read with TypeScript's own JSONC reader rather
than by stripping comments: in a tsconfig "@/*" opens what a block comment
pattern reads as a comment, and the */ in a later glob such as
"**/node_modules" closes it, so everything between them disappears. That
silently took paths with it and left @/ mapped to the project root, which
is a passing typecheck and a suite that cannot resolve its own imports.
kiln build emits dist/server.mjs, which is the generated entry, and
kiln start runs it. An application that keeps its own src/main.ts is
honoured instead.
kiln routes is the one worth knowing about: it lists every discovered
route with the method, the handler file and what the caller must hold.
GET /spaces/:id Space/get.ts spaces:read
POST /spaces/:id/share Space/share.ts spaces:manage
POST /auth/login Auth/login.ts publicOverriding any of it is optional, in a file that imports nothing - separate
from app.config.ts, which imports application code and so cannot be read
before the application is built:
// kiln.config.mjs
/** @type {import("@kilnjs/core/toolchain").KilnToolchain} */
export default {
entry: "src/server.ts",
build: { external: ["sharp"] },
esbuild: (options) => ({ ...options, minify: true }),
transformEsm: ["sanitize-html"],
};The annotation is a block comment on purpose, and kiln/no-block-comment
exempts a one-line one: see below, under the lint rules.
What the runner transforms
transformEsm names a package the suite loads that cannot be required as it
is, and what that package drags in comes with it. sanitize-html is
CommonJS and depends on an ESM-only htmlparser2, which depends on
entities, domhandler, domelementtype, domutils, dom-serializer and
nanoid, some of them as nested copies: each one used to surface as its own
"Must use import to load ES Module" failure, one run at a time, until all
seven were written out by hand. The closure under a declared name is walked
from the application's own node_modules, the way Node resolves it, and
every package whose manifest says type: module is added. A declaration is
now the package somebody installed.
The transform itself reads .mjs and .cjs as well as .ts and .js,
because a lint config is one of those and a suite that imports it otherwise
fails on the same "Must use import to load ES Module".
What it provides
| Subpath | What is in it |
| ------------------------------ | --------------------------------------------------------------------------------- |
| @kilnjs/core | startServer, buildServer, CONFIG, the caller type |
| /modules | KilnModule, the contract a module implements |
| /endpoints | Endpoint, ValidationSchema, ControllerSettings |
| /discovery | mapEndpoints, for asserting every endpoint declares its intent |
| /middlewares | Auth, authorize, requireLocal, handleWebhook |
| /errors | GenericError and the typed subclasses. Reaches no configuration |
| /database | withCacheInvalidation, hotReloadSafe |
| /redis | The Redis connections the rate limiter, the cache and the drain use |
| /security, /identity | The vault, the tokens, the authentication hook |
| /context | Request-scoped AsyncLocalStorage |
| /application | The discovery primitives, AppConfig and its extension points |
| /environment | defineEnvironment, which is where an application's own variables are read |
| /registry | buildLazyHandle, buildNamedFacade: how a package resolves its own named implementations |
| /seed | The ordered seed runner |
| /bootstrap | bootServer, the entry point the CLI runs |
| /generated | linkGeneratedPackage, how a generator declares what it wrote |
| /uploads | readUploadFromRequest, readUploadsFromRequest: a multipart file, checked. Needs handler.uploads |
| /telemetry | reportTelemetryError, and the reporter a module registers |
| /toolchain | KilnToolchain, the type of an optional kiln.config.mjs |
| /eslint, /prettier, /jest | The lint, format and suite configuration a project spreads |
| /tsconfig.base.json, /tsconfig.emit.json | The compiler options a project extends |
| /config, /utils, /server | The rest |
The generated database client is not one of these. It belongs to the
application, and kiln db generate writes it and links it as
@kilnjs/generated: /client, /models and /model-extensions are that
package's subpaths. See below, under the generated client.
An upload is held to its bytes
readUploadFromRequest checks the accept list against the type the client
claimed, which is all a multipart part carries, and then against what the
leading bytes say where they say anything. A file whose bytes are a PDF is
refused by an endpoint that takes images, whatever the part called it, and a
JPEG sent as a PNG is refused even where both are accepted: stored under the
wrong type it is served under the wrong type, and a type a browser gets wrong
is how a stored file becomes a script.
A format with no signature carries nothing to check and is still taken on the
claim alone, which is most text formats, so verifyContents: false is for an
endpoint taking a format this does not know rather than a way to go faster.
The filename is handed over as the parser gave it, which is a basename:
busboy takes one unless told otherwise. It can still be .., and it is still
a string somebody typed, so an endpoint generates its own storage key and
keeps the filename as data.
What a request costs
scripts/benchmark.mjs reports CPU microseconds per request against a
server booted from this package's own fixture application, which reaches no
database:
KILN_ROOT=$PWD/tests/application NODE_ENV=test \
node --import tsx scripts/benchServer.mts &
node scripts/benchmark.mjs 4500 <that pid>To measure what a module costs, name it rather than editing the script: core is module-agnostic, and a benchmark is not the exception. Its options come from its own environment variables.
BENCH_MODULES=@kilnjs/metrics METRICS_PROCESS=false \
KILN_ROOT=$PWD/tests/application NODE_ENV=test \
node --import tsx scripts/benchServer.mts &CPU rather than throughput, because throughput on a shared machine is the scheduler's number: three runs of one configuration spread by ten per cent, which hides every change worth making. The server's own counters hold to about half a microsecond over twenty thousand requests.
| | | | --- | --- | | public GET, one row | 25.0us | | public GET, a hundred rows | 40.5us | | guarded POST, a validated body | 39.5us |
The table in tests/server/performance.test.ts breaks those down into what
the framework adds, each figure measured by taking that one thing out with a
baseline beside it. Two of the entries there are things that look like wins
and measured as losses, which is the point of writing them down.
LEGACY_SECURITY_HEADERS=false is the one dial. Six of the twelve security
headers are read only by a document, a dead plugin or a retired browser;
leaving them out saves about 1.5us of a 25.5us response and changes nothing
about how a JSON answer is treated. On by default, because what a deployment
serves is not core's to narrow.
What the environment has to be
An application that fails to boot in production usually fails on the environment rather than on its own code: a variable nobody set, a service one major behind what the code assumes, a host unreachable from where the process actually runs. None of that is visible from the source, so it is learned one failing deploy at a time.
So each module declares what it needs, and the needs are data. kiln env
reports them against what is actually here:
SERVICES
ok postgres 16.15 (needs 14) from DATABASE_URL
Reachable
ok redis 7.4.11 (needs 7) from REDIS_URL
Reachable
FINGERPRINT
e127e9771e8e the same on two boxes means the same environmentA refusal and a warning are not the same kind of thing, and the line between them is the design decision here. Refuse when the difference changes behaviour the code already assumes. Warn when it only changes performance or confidence.
An optional variable that is absent is neither. It is the configuration
the declaration said was survivable, so it is reported as met and says it is
unset. A boot that warned about every one of them printed seven warnings for
an application with nothing wrong, which is how the one that mattered got
missed. A declaration whose absence is survivable and worth saying every
time asks for that with optional: "warn", the same word defineEnvironment
takes.
A variable belongs to whoever reads it. Core declares none of a module's:
SENTRY_DSN and SENTRY_RELEASE are @kilnjs/telemetry's, so an application
without that module installed is never asked for them. @kilnjs/auth declares Redis 7 because the
sign-in lockout counts with EXPIRE NX, which arrived in 7: below that the
account window silently slides and a named account can be locked out
indefinitely by anyone who can spell its address. That reason lived in auth's
README for months, where no boot could read it. An author adding a check
applies the same test rather than filling in a boolean, which is why neither
word is a field a declaration sets.
Unreachable is usually a warning, and absence is sometimes worse than
failure. Core declares Redis without fatal, because the rate limiter is
written to fail open and a server refusing to boot without its cache turns a
cache rollout into an outage. @kilnjs/jobs declares the same service
fatal: true, because a queue nobody is running accepts work and never does
it, and silence is the failure. A probe that runs out of its budget is
reported as unknown rather than broken: a boot that refuses because a DNS
lookup was slow is its own outage. Every probe runs concurrently and the
budget is each service's own, so an SMTP host that never resolves no longer
reports a Postgres that answered in 11ms as unprobed, which is what a shared
budget did.
What core itself reads is configurable too, and every one of these is
declared where it is read, so kiln env prints it with its default:
| | |
| --- | --- |
| LOG_DIRECTORY LOG_RETENTION_DAYS | where the rolling log files go and how many days are kept |
| LOG_REDACT_NAMES LOG_REDACT_DEPTH LOG_REDACT_WIDTH | what else is never written, and how much of a record is kept |
| RATE_LIMIT_NAMESPACE | what every counter sits under, so two deployments on one Redis do not share a budget |
| SHUTDOWN_DRAIN_MS | how long a drain has before the process exits anyway |
| WEBHOOK_SIGNATURE_HEADER WEBHOOK_TIMESTAMP_HEADER | the names a sender's signature arrives under |
LOG_REDACT_NAMES is the one worth knowing about: core takes out anything
reading as a password, a secret, a token, a key, a DSN or a one-time code,
and an application knows about fields it cannot guess at, such as an account
number or an address. Each name matches anywhere in a field's own name, so
iban covers customerIban.
The report is worth more than the gate, so it is built to be pasted into an issue. Nothing prints a value. A connection string is reduced to its shape, with the user kept and the password never; anything whose name reads as a credential is reported as set and its length. The fingerprint is a digest over what is identifying rather than what is current, so it does not change when a value does and carries nothing in it.
The fingerprint is a digest of a list, never the decision. The migration
gate works because its fingerprint is named items and the failure says which
ones are missing. A single hash that failed a boot over a patch bump would
teach everyone to set the skip flag, and then nothing is checked while
everyone believes something is. Every refusal names its own item;
KILN_SKIP_ENV_CHECK=1 overrides the gate, which is a decision worth writing
down.
And the list reads backwards. kiln env --compose and --env-example
generate the two files that are otherwise hand-maintained and therefore
wrong, from the same declarations, with each service at or above the version
it was declared with. Neither is applied: the operator reads the output and
commits it, the way the migration gate prints a command rather than running
one. A value appears in the generated example only where a declaration gave
one that is safe to commit, because that file lands in a repository.
kiln check runs kiln env --offline, which checks the declarations and the
variables where no service is reachable. A container build has neither a
database nor a Redis, and a gate that refused every image build would be
turned off within the week.
An application's own variables
CONFIG holds what core and the installed modules declare, and
process.env is a lint error. So a variable belonging to the application
itself had nowhere to be read from, and every project began by disabling the
rule. defineEnvironment declares it and reads it in one move:
// src/config/index.ts
import { defineEnvironment } from "@kilnjs/core/environment";
export const ENV = defineEnvironment({
INBOUND_DOMAIN: {
kind: "string",
because: "a reply address is built under it",
},
REPLY_SECRET: {
kind: "string",
min: 32,
secret: true,
because: "a reply token is signed with it",
},
PROVIDER_SECRET: {
kind: "string",
optional: true,
because: "the inbound provider signs its own notifications with it",
},
});Nothing is registered by that call: app.config.ts hands it over as
environment: ENV, which is what puts the declaration in front of the gates.
ENV.INBOUND_DOMAIN is a string, ENV.PROVIDER_SECRET is
string | undefined, and the kinds are the ones core's own configuration is
built from: string, number, integer, boolean, url. The same
declaration is what kiln env asks for, with the because printed beside
it, and what --env-example and --compose are generated from.
What each kind reads as is written out rather than inferred from the
schemas. z.infer over them emitted z.z.ZodCoercedNumber into the
declaration, a type reachable only through a namespace inside zod's own: the
compiler resolved it and the lint program did not, so ENV.PORT was a
number to tsc and an error type to no-unsafe-argument, and
Math.max(ENV.PORT, 0) was a lint error nobody could answer. The integration
application lints that exact line, which is the only place the two programs
can be seen to disagree.
A required variable that is missing is thrown where it is declared rather than warned about: a handler reading one that is not there fails on the first request, somewhere nothing points back at the cause.
The output needs nothing installed
kiln build bundles every dependency, copies native files in beside the
output and leaves only tsx external, which nothing in a served request
reaches for. So the output directory runs on its own: copy it into an image,
run node server.mjs, and there is no install step and no lockfile to
honour. The build writes a package.json into the output saying so, with the
entry, the engine floor and kiln.standalone.
That was true and undeclared for a while, which is the same thing as
accidental: one dependency resolving a package by name at runtime would
quietly end it and only a deploy would find out. A static scan cannot answer
it either, because the bundle contains bare specifiers that never execute
(ajv's code generator emits paths as strings) and optional peers behind a
try/catch. So the framework's own suite copies the output somewhere with no
node_modules above it and starts it.
A module may say the bundle cannot inline something, in kiln.external in
its own manifest, and then the output is honest about it: those packages are
left external, written into the built package.json as dependencies at the
ranges the application asked for, listed in kiln.external, and
kiln.standalone is false. @kilnjs/next is the case: the server imports
next, and Next imports react and react-dom from a node_modules beside
itself, so a build with a front end declared is not standalone and used to say
it was. An image then failed at boot with ERR_MODULE_NOT_FOUND rather than
at build.
A module may also build its own half, in kiln.build, which kiln build
runs after the bundle. kiln next build compiles the pages, so an application
with a front end has one build command rather than two that have to agree
about where the output goes.
A generator runs without the environment a server needs. kiln generate
runs each module's generator with KILN_ENV_PLACEHOLDERS=1, under which a
missing variable reads as empty rather than stopping the process: generating
reads discovery, discovery loads app.config.ts, and that reads every module's
settings, so a Docker build stage was refused for want of REDIS_URL,
VAULT_KMS, JWT_SECRET and COOKIE_DOMAIN, none of which it uses. A
server refuses to boot in that mode, which is what makes it safe to have:
the failure is structural rather than a note in a README.
A module's own build runs in that mode too, for the same reason: it loads
app.config.ts to find out what to build. It is set on the child process
rather than inside it, because an import of the configuration is hoisted over
any assignment the child's own entry makes, so by the time the assignment ran
the environment had already been read and refused.
Two things used to defeat the mode from inside. security/crypto hashed
VAULT_KMS into a key as it was imported, which threw on undefined whatever
the mode said; the key is now derived on first use, and nothing a generator
does encrypts anything. And auth() read its settings at the declaration, so
every command that loaded app.config.ts asked for JWT_SECRET and
COOKIE_DOMAIN; a module reads its settings in setup instead, which still
runs at the boot and makes a bad secret a boot error.
The version the output reports is set in the bundle's banner, as
globalThis.process.env.APP_VERSION ??= "...", before any import runs. Through
the bare name it read the wrong process: a bundle is one module scope, and
html-to-text (which the mailer pulls in) declares a top-level function
process that is hoisted over the global, so the built server threw on its
first line.
The gates can be skipped one at a time. KILN_SKIP_GATES=1 is still all
of them; KILN_SKIP_GATES=audit or audit,knip is the ones named, because a
build that cannot reach a registry should not also give up its schema and dead
code checks. Either way it says what it skipped.
What is checked instead of remembered
Nine mistakes used to be knowledge. Each of them had shipped in an application built on this, each was documented somewhere as a thing to remember, and remembering is not a mechanism.
An answer that carries more than it promised. Returning a database row
compiles, because a row is a supertype of the declared response rather than a
literal, so every column goes out: one application's user list sent
invite_expire_date and updated_at to anybody who could read it. Outside
production the answer is now compared to its schema and the extra fields are
named with their paths. In production nothing is compared and the handler is
bound bare, which is the trade the framework made deliberately:
User/list answered with 2 field(s) it did not promise:
[].invite_expire_date, [].updated_at. Build the answer you mean, or name them
in the responseA response schema naming a type JSON cannot carry. z.date() infers
Date and arrives as a string, so five fields across two applications
promised their callers something they could never receive. That is a property
of the declaration rather than of any answer, so it is refused where
declarations are read, which means kiln routes and kiln openapi refuse it
too. A lint rule says the same thing in the editor, because the fix is one
word.
A methods directory with no index.ts. The generator finds no methods and
attaches nothing: not an error, not a warning, just a delegate with nothing on
it until a handler calls one. And a method named after a Prisma one, which
replaces it, so the call inside reaches itself and the stack overflows a long
way from the name that caused it. Both were documented traps. kiln models
is now a step in kiln check.
A module's discovered directory. knip's entry list was a hardcoded array
in core, so a module shipping a segment of its own needed an edit to core
before an application using it could pass its own gate: the audit and payments
entries were added exactly that way, after both had shipped. Each module
declares its own in kiln.segments in its manifest, which plain node can
read, and the list is derived from what is installed.
The application's own kiln.segments is read the same way, for a
directory it discovers itself through loadModulesFromSegment: nothing else
can know about one, and the alternative was a hand-written knip.json, which
replaces the generated one whole and so has to be updated by hand every time
a module with a segment is installed.
A response promising a credential by name. The boot already refused a
column whose name reads as a stored credential and which nothing omits, which
closes the way a secret leaves by accident: a row returned whole. A response
schema naming password or invite_token is the other half, and it was open.
It is not an accident at all, it is a promise, and the omit list cannot stop
it: a handler that selects the column deliberately sends it deliberately. So
discovery refuses it, which means kiln openapi cannot publish a document
saying a credential comes back.
Some of them are meant, so the endpoint names the field:
handler.returnsCredentials = ["invite_token"];An invite token exists exactly once, at the moment it is issued, because only its hash was kept. A check with no way to say that is a check people satisfy by renaming the field to something that does not read as a credential, which is worse than no check. Per field rather than a flag, and held to the response: a name there that the response no longer promises is refused too, because an exemption for a field that is gone is one nobody knows they still hold.
Both checks read the type as well as the name. A number, a boolean, a
BigInt or a date cannot carry a credential whatever it is called, so
recovery_codes_remaining: z.number() is a count rather than a promise, and a
relation is not a column at all: recovery_codes RecoveryCode[] holds nothing,
returns nothing unless a query includes it, and omit cannot name it, so the
boot asking for it to be omitted was asking for something Prisma refuses. A
check that fires on a name alone is one people satisfy by renaming the field,
which is worse than no check.
A cross-origin write carrying a session cookie. There was no CSRF defence
at all. The only thing between a cookie session and a forged write was
sameSite: "lax" on the cookie, and lax is scoped to the site:
evil.example.com and api.example.com are one site, so anything holding any
subdomain of a deployment could post as the caller with the caller's own
cookie attached. CORS is not the answer and never was, because CORS decides
who may read an answer and a forged write does not need to read it: the
request is sent, the write happens, and the browser reports a CORS error to a
console nobody is watching.
An unsafe method carrying a cookie is now checked against Origin, with
FRONT_END_ORIGIN and the server's own origin as the list. A request carrying
no cookie is not checked at all, which is what keeps every token
authenticated client working from anywhere: a bearer token is attached by
whoever holds it rather than by a browser that merely visited a page. The hook
is installed only on routes that can be forged, so a GET carries nothing for
it. handler.allowsCrossSite = true opts out, and kiln/no-cross-site makes
that say why.
A write behind a safe method. The check above exempts a safe method, and
it has to: a read exercises no ambient authority, and paying a hook on every
GET to discover that would be the most expensive thing in the request path.
The exemption is only sound while a write is never behind a safe method, and
nothing held it to that. httpMethod defaults to GET, so a file in
mutators/ that forgot the declaration was mounted as a GET: it writes, a
top-level navigation reaches it, lax sends the cookie on exactly that, and the
forgery check skips it for being a read. A link in an email was enough.
Discovery refuses it now, so kiln routes does too.
A request in an error's context. The house rule said "never put a whole
request, body or headers in an error context, it is reported as it is" for as
long as the error model existed. What checked it was a selector matching a
property whose key was named body, params or req, which caught
context: { body: req.body } and nothing else: the key is not where the leak
is. context: req.body, context: req, context: { payload: req.body } and
context: { ...req.body } all passed it. kiln/no-request-in-context reads
the value instead, and leaves a named field alone, including one header.
The redaction that already existed is a backstop, not permission. It matches
names, so a token under a name it does not recognise still goes out, and every
other field of a body goes out regardless: the caller's data rather than a
secret, and no better for being sent. The same vocabulary now covers the
logger, which was constructed with no redaction at all, so
req.log.info({ body: req.body }) wrote a password to the log file in the
clear.
TRUST_PROXY set to a hop count. Everything keyed on who a caller is
reads req.ip: the rate limiter buckets anonymous traffic by it, the sign-in
lockout counts attempts by it. Where the proxy in front is not trusted,
req.ip is the proxy's address on every request, so the whole internet shares
one bucket and one lockout counter and a single sign-in attempt locks out
every account.
Fastify documents a hop count, its types leave it out, and the code is the
thing: a numeric trustProxy returns a trust function answering false to
every peer, on purpose, because a hop count cannot validate the peer a request
came from. Measured on 5.12.5, trustProxy: 1 with an X-Forwarded-For of
203.0.113.7 reports req.ip as 127.0.0.1. So a hop count is not a weaker
setting, it is no setting at all, written by somebody who believed they had
configured one. It is refused at boot, with true or an address list as the
answer.
Whether the setting matches reality cannot be answered at boot, because
nothing in the configuration says what is in front of the process. So the
first two hundred requests are counted, and if more than nine in ten carried
X-Forwarded-For and it was ignored, that is said once with what it costs. A
handful among many is somebody poking at the server and ignoring those is
exactly right, which is why it is asked of a sample rather than of the first
request that looks odd.
Two more are smaller and were just as real. kiln migration --name=x writes
the timestamp, because a hand-written migration is the only kind whose name a
person chooses and one chosen badly sorted before the schema it depended on.
And getPrefixForController is exported, because it existed, was reachable
from no subpath, and two printers carried their own copy of it: a printed path
nothing serves is bad enough in kiln routes and worse in kiln permissions,
which proves sentences about paths.
A webhook authenticates by signature
A route another service calls has no session and no origin, so it carries a
signature instead. The sender signs <timestamp>.<body> with WEBHOOK_SECRET
and sends the hex digest in x-kiln-signature with the timestamp in
x-kiln-timestamp; the route declares one thing:
handler.verifiesSignature = true;Core then keeps that route's body as it arrived and verifies the digest in
constant time, refusing a timestamp more than WEBHOOK_TOLERANCE seconds out.
Every refusal is a 404, so the route cannot be found by probing it.
The signature is over the bytes, not over the parsed body. Re-serialising what the parser produced does not reproduce what was signed, so the bytes are kept, and only for a route that declares this: they are held for the length of the request and no other route pays for them.
The check runs after the body is read and before it is validated. An
unverified caller is answered 404 rather than a validation error naming a
field, and handleWebhook in onRequest is refused at discovery, because
that phase runs before the body exists and would refuse every caller.
One secret guards every route that declares it. A sender that must have a
key of its own verifies it in the handler, over req.rawBody.
The two header names are the deployment's. A sender that writes its own,
as GitHub does, is read with WEBHOOK_SIGNATURE_HEADER and
WEBHOOK_TIMESTAMP_HEADER; core's own names are the defaults and stop being
a second way in once either is set. sha256= in front of the digest is
accepted either way.
Who may read an answer
CORS was one origin, from one variable, with credentials unconditionally on:
fastify.register(cors, {
origin: CONFIG.SECURITY.FRONT_END_ORIGIN,
credentials: true,
});That is right for a server with one dashboard in front of it and cannot
describe an API a third party calls from a browser, where the origins it may
be called from are the application's own data, a row per client. Worse, the
plugin answers the preflight itself, so an application's own OPTIONS
endpoint never runs and there was no seam to work around it from.
So the decision is the application's, in src/cors.ts, discovered the way
src/auth.ts is:
import type { DecideCors } from "@kilnjs/core/security";
const decideCorsForRequest: DecideCors = async (req) => {
const origin = req.headers.origin;
if (origin === undefined) return null;
const channel = await database.channel.findByOrigin(origin);
// No credentials: this API authenticates by key, and a cookie it never
// reads is one a browser could be made to attach
if (channel) return { origin, credentials: false, maxAge: 600 };
return null;
};
export default decideCorsForRequest;Answering null means core's own answer, so a decision function defers
rather than restating the dashboard's origin, and an application that
declares no src/cors.ts behaves exactly as before. credentials is off
unless asked for, because a cookie session and a third-party origin is the
combination that forges writes, and * with credentials is forbidden by the
spec anyway. origin: false sends no CORS headers at all, which is the
refusal a browser enforces. preflight: false hands OPTIONS to the route
for an application that answers it itself.
A decision that throws is not a decision to allow: no headers go out and the
reason is logged here rather than sent. And none of this is the forgery
check: that reads Origin on an unsafe method carrying a cookie, which CORS
cannot see, and is still core's.
An empty body
$ curl -X POST .../webhook/verify -H 'content-type: application/json'
{"message":"Body cannot be empty when content-type is set to 'application/json'"}Fastify refuses an empty body whenever the content type says JSON, in the
parsing phase, which is before the route's own schema or its permission is
consulted. Every HTTP client, SDK and fetch wrapper sets that header by
default, so an endpoint taking no body was unreachable without unsetting it
by hand, and no API documentation anywhere says to.
Core replaces that parser. An empty body is undefined, so an endpoint whose
schema declares no body answers, and one that declares a body is refused by
its own schema instead, which is what makes the refusal name the field. A
body that is not JSON is a 400 saying so.
It also fixes an ordering nobody meant: the refusal came before
handler.authorize, so a caller holding nothing was told "Body cannot be
empty" rather than 403, and an authorization test passed for the wrong
reason.
The walls
Core owns what a build and a commit are held to, so a wall moves with the framework rather than with whoever remembers to copy it.
| | |
| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| kiln toolchain | more than one version of TypeScript installed anywhere in the tree, which is not a vulnerability and is worse than one. Runs first, in kiln check, because it is what the compiler and the linter read types through |
| kiln knip | anything nothing reaches: a file, an export, a dependency. The configuration is generated, because what is reached by discovery rather than by an import is core's to know |
| kiln audit | a known vulnerability in anything installed. The tool follows the lockfile, because each package manager audits only its own |
| kiln check | every wall in order, including each package's own verify, and only the ones whose inputs moved since they passed |
| kiln hooks | writes .husky/pre-commit and .husky/pre-push to run them |
The suite rebuilds its database itself, rather than through
prisma migrate reset: it drops and recreates the schema and then runs
migrate deploy, which is the same end state, applied from the migrations so a
suite fails the way a deploy would when one is missing. Two things make that
safe, and both run before anything is dropped: the suite refuses outside
NODE_ENV=test, and it refuses a DATABASE_URL matching the one .env names.
Those are stronger than a heuristic about who is running it, which is the other
reason for doing it this way: Prisma 7 refuses migrate reset whenever it
detects an AI agent and asks for consent carrying the user's own words, so
every test run under one needed a variable set by hand.
The suite runs four workers over one test database, which is why the cache
is scoped per worker. KILN_JEST_WORKERS sets the count for an application's
suite as well as for a package's, so a test about state the whole database
shares can be made writable with KILN_JEST_WORKERS=1 without taking the test
step out of kiln check.
kiln check remembers what passed. Each step records a fingerprint of
the directories it reads into .kiln/check.json, and a step whose inputs have
not moved is not run again. The fingerprint includes core itself, resolved
through the link a project reaches it by, so a rebuilt framework re-runs
everything. A step that fails has its entry dropped rather than recorded, and
audit is never cached because its input is an advisory database elsewhere.
--no-cache runs the lot.
Pins
The framework holds a few dependencies at a version and kiln pins writes
them into the application's manifest, under resolutions or overrides
depending on the package manager. kiln audit fails until they are there, and
a module declares its own in kiln.pins.
knip's optionalPeerDependencies rule is off, for the same declaration:
a module marks fastify and zod optional because core installs them and the
application is told never to, and then uses both. knip reads a used optional
peer as a mistake, which it is where the application is the supplier and is
not here.
A pin is how an advisory in the toolchain is answered, because an
application cannot write one itself: resolutions belong to the root of the
install and kiln pins is what writes there. js-yaml is pinned to 4 for
that reason. Jest reaches @istanbuljs/load-nyc-config, which asks for
js-yaml 3, whose argparse 1 brings a sprintf-js with a denial of
service and no fixed version published. It runs only under coverage and never
in a served request, which makes it the kind of advisory nobody can act on
and everybody has to read; 4 reads the same configuration through the same
load, and the audit is clean.
Everything that reads the install finds its root by walking up to the
nearest lockfile: the pins, the audit, the installs kiln setup runs, and
the toolchain count. An application inside a workspace is therefore pinned,
audited and counted where its dependencies actually are. The count is the one
that hid the longest: looking only in the application's own directory, it
found one copy of three, printed a pass, and left the compiler disagreement
below it to surface as about ninety lint errors that pointed nowhere near the
cause. They used to disagree: pins walked up and audit looked only
in the application's own directory, so in a workspace member the audit ran
npm audit against no lockfile and failed with ENOLOCK. Symlinking the
lockfile down then made the audit read the member's manifest for pins that
were written into the root's, and refuse for their absence.
TypeScript is pinned to one copy, and that is not about a vulnerability. A
typed lint rule asks TypeScript about a type through ts-api-utils, and both
ts-api-utils and eslint-plugin-sonarjs will resolve their own TypeScript
if nothing stops them. A rule then reads a type built by one instance through
another instance's flag constants, and the answers are wrong in the worst
way: not an error, a confident falsehood. Measured on a fresh application,
with two copies installed:
| | |
| --- | --- |
| given.failure?.slice(0) where failure?: string | "Unnecessary optional chain on a non-nullish value" |
| address.startsWith("fc") where address: string | "TypeError can be thrown as address might be null" |
| state !== "VERIFIED" on a two-value enum | "Unnecessary conditional, value is always falsy" |
Ninety such errors in one application, every one of them false, and nothing about them says the cause. One pin removes all of it.
An application that installs a second copy anyway has no way to know, and
kiln audit only looks for advisories, so kiln check counts them. It is
the first step, it names every copy and where it is, and it says to run
kiln pins. Two copies of the same version are not a disagreement and are not
refused: a duplicate install is wasteful rather than wrong.
kiln build runs the first two itself, after validating the schema, so a
bundle cannot be produced from code that would not pass. KILN_SKIP_GATES=1
is for iterating locally and says what it costs.
Releasing
kiln publish # patch whatever changed
kiln publish --dry-run # say what it would do
kiln publish --bump=minorEach package is versioned on its own, because they are released on their own: a fix in the mailer is not a reason for an application to take a new core.
What counts as changed is answered against the registry, not against git. The digest of the files npm would pack is written into the published manifest, so a release compares this working tree to what consumers actually have: no tags to keep, no history to have, and a fresh clone answers the same.
The digest ignores the version and the digest field itself, so stamping a
release is not a change, and it covers everything that ships, so a README
edit is. It is written to kilnDigest, beside kiln rather than inside it:
kiln holds what a package declares to the toolchain, written by a person,
and a release that assigned into it once replaced the lot. A generated value
and a hand-maintained one do not share a key. The version is counted from what is published rather than from what
is local, which means a hand edit to package.json cannot skip a number or
reuse one.
Packages are published in dependency order, the walls run first, and a failure stops the run with everything before it already published, which is the honest state: running it again finishes the job.
If the digest cannot be read back for any reason, the package reads as changed and is published again. Publishing twice costs a version; publishing nothing and reporting that it shipped costs an afternoon of debugging against stale code.
Migrations, which a build carries and a boot checks
A server whose database is behind its code answers wrongly rather than loudly: a missing column fails on the one request that needs it.
So kiln build writes the migration history's fingerprint into the output,
and the boot compares it to what _prisma_migrations says has been applied.
Anything missing and the server refuses to serve, before a route is mounted.
It needs no shadow database: it reads the directory and the table Prisma
already keeps.
KILN_SKIP_MIGRATION_CHECK=1 turns it off for a deliberate deploy ahead of
a migration, and warns.
An endpoint says what it answers with
schema.response is required of every endpoint, by lint and by type:
export const schema = {
body: z.object({ email: z.string().email() }),
response: z.object({ id: z.string(), email: z.string() }),
} satisfies ValidationSchema;
const handler: Endpoint<typeof schema> = async (req) => { ... };The handler's return type is checked against it, so a field removed from a model is a compile error at every route that promised it. That is where the guarantee lives: nothing re-reads the schema at runtime.
A handler's two arguments are KilnRequest and KilnReply, both exported
from @kilnjs/core/endpoints. The rule set forbids importing fastify to
reach FastifyReply, so a helper taking the reply had no type to ask for and
was reading it off Parameters<Endpoint<ValidationSchema>>[1].
An endpoint file reads in one order, held to it by
kiln/endpoint-file-order: the imports, the handler, its method and path and
everything small beside them, its permission, its schema, the default export.
The small declarations ranked together above authorize are httpMethod,
path, isPublic, isHidden, uploads, allowsCrossSite,
returnsCredentials, isExternal, onRequest and preHandler. The refusal
names the declaration that moved and where to put it, because naming only the
group it belongs to said nothing: "its method and path comes after its
permission" mentions neither returnsCredentials nor where it goes.
It used to. Parsing every answer on the way out cost roughly four times what
serialising it costs and grew with every row, which on a list endpoint was
the largest single cost in the request path, all to re-establish something
the compiler already knew. What the change leaves to the handler is the one
thing the parse also did by accident: an answer carrying more than it
promised now goes out with all of it. A column stays out of a response by
not being read, or by the omit list in app.config.ts, rather than by being
filtered on the way past.
So the omit list is the one mechanism that survives somebody else's code,
and the boot checks that it is actually used. Every column whose name reads
as a stored credential (password, *_secret, *_token, api_key,
recovery_code, salt) has to be either excluded there or named in
database.exposed:
1 column(s) read as a stored credential and are returned by every query that
does not exclude them: user.invite_token.
Add each one to `database.omit` in app.config.ts, or, where it is meant to be
readable, to `database.exposed` with a comment saying why.Not a warning. A warning is read once, by whoever was watching the log the day the column was added, and the failure it precedes is a credential in every response that returns a user: the compiler is happy, because a row is a supertype of the declared response, and the test asserting the fields that matter passes. The refusal is the same trade the migration gate makes.
The question is deliberately narrow. Core cannot know whether a column is
sensitive, so it asks whether the name reads as a credential, which is a
judgement about English and is nearly always right. The vocabulary is
narrower than the one core redacts out of a telemetry report for the same
reason: a false positive there costs a redacted value in a report, and here
it costs an application its morning. key on its own is not in it, because
storage_key is a path; auth is not in it, because it is half the words in
a schema with sessions in it. An entry in database.exposed naming a column
the schema no longer has is refused too, since an allow list nobody can check
is how the next one gets added to it.
A BigInt goes out as a string. JSON.stringify refuses a BigInt and the
prototype is its only hook, so core installs one in
server/serialisation.ts. It used to convert to a number, which is silent
corruption above 2^53: an id of 9007199254740993 went out as
9007199254740992 and a Telegram channel id of -1001234567890123456 went out
as -1001234567890123500, in a response that looked perfectly well formed.
A BigInt column exists because the value does not fit in a double, so
converting it to one answers the wrong question. The consequence to know is
that a z.bigint() response field is typed bigint and arrives as a string,
so a schema that means it should say z.string() and the handler should
convert.
The document, and what an endpoint can say in it
kiln openapi builds it from those same schemas, so the document says what
the code declares. Three things an endpoint declares for the reader rather
than for the server:
handler.describe = "Accepts one message for a channel";
// How a caller authenticates, where it is not the session cookie: a public
// API's routes are public to core and still refuse an unknown key
handler.credential = { kind: "bearer" };
handler.credential = { kind: "apiKey", in: "header", name: "X-Api-Key" };describe becomes the operation's summary, and a controller's own
describe becomes its tag's description. A declared credential becomes a
security scheme and that route's security, and adds the 401 a public route
would otherwise not be documented as answering. Core reads isPublic for
the rest: a guarded route carries the cookie scheme, and a public one with
no credential carries none, which is the one case worth stating outright.
The builder is @kilnjs/core/openapi, so an application can serve its own
document or assert on it in a suite rather than spawning the CLI.
include narrows which endpoints are in it, by whatever they declare, since a
dashboard's routes and a public API's are two documents:
import { buildOpenapiDocument } from "@kilnjs/core/openapi";
const document = buildOpenapiDocument(controllers, {
include: (endpoint) => endpoint.isExternal === true,
});What the boot registers, and what it does not
Three things a server carries are paid for on every request, so two of them are conditional on something asking:
| | |
| -------------------- | --------------------------------------------------------- |
| the security headers | always, from a frozen object computed once |
| the multipart parser | only where an endpoint declares uploads |
| the rate limiter | only where it is enabled, and it costs a Redis round trip |
The multipart parser is the surprising one. It puts thirteen properties on every request and initialises four of them per request, which measures at about a quarter of this server's throughput once it compounds with the other hooks. An application that takes no uploads should not pay for the ones that do, so the boot reads the discovered endpoints and registers it only where something declared an upload. Where nothing did, a parser is still registered for the content type and all it does is explain itself: without one, Fastify would answer 415 and the reason would be a guess.
The security headers were the same shape of problem. @fastify/helmet runs
a middleware per response to arrive at a set that, for this configuration,
never changes: no nonce, nothing read from the request. Asking it once at
boot and setting the answer from a frozen object is worth about forty per
cent of the throughput, for byte-identical headers.
What one caller may spend
Two ceilings, because the anonymous surface and the authenticated one are not
the same risk: RATE_LIMIT_AUTHENTICATED_MAX is registered globally and keyed
on the caller, and a public route is mounted with RATE_LIMIT_MAX instead,
keyed on the address.
An endpoint whose risk is not either of those declares its own, because a sign-in form and an intake endpoint another server calls for every one of its users are not the same question and one number cannot answer both:
handler.rateLimit = { max: 600, window: 60_000 };
// What the budget is counted against, where an address is the wrong unit:
// one server calling for all of its users is one address
handler.rateLimit = { max: 600, by: (req) => readApiKeyFromRequest(req) };
// Or out of the limiter altogether, which something else is holding
handler.rateLimit = false;window is milliseconds and falls back to RATE_LIMIT_WINDOW. A max below
one is refused at boot: it answers 429 to everybody and nothing says why.
The limiter as a whole is server.rateLimit in app.config.ts, over
whatever the environment says, and the boot holds what it read so the routes
mounted after it are mounted with the same numbers:
server: {
rateLimit: {
max: 1200, // Per caller, per window, on a guarded route
publicMax: 60, // Per address, on a public one
window: 60_000,
namespace: "mailman-rate-limit:", // Two deployments, one Redis
allowList: ["10.0.0.4"], // A health checker that is not a caller
key: (req) => readTenantFromRequest(req), // What a caller is counted as
},
},Each of max, publicMax, window, namespace and enabled has an
environment variable behind it, so a deployment can move one without a build.
allowList and key are functions and are the application's to pass.
What the parser and the drain are given
| | |
| --- | --- |
| server.uploads | every multipart limit, not only the file size: fileSize, files, fields, fieldSize, fieldNameSize, parts, headerPairs |
| server.shutdown | drainMs, and the signals that start a drain |
| server.headers | over the security headers core works out. A string replaces one, false drops one |
A form carrying more fields than core's hundred is a 413 nobody can explain,
and a platform that gives a container longer than five seconds to drain, or
sends something other than SIGTERM, is the reason the other two exist.
server.headers is for a proxy that sends its own HSTS, or an application
that needs a content security policy of its own:
server: {
uploads: { fields: 400, parts: 410 },
shutdown: { drainMs: 25_000, signals: ["SIGTERM"] },
headers: { "strict-transport-security": false, "permissions-policy": "camera=()" },
},The lint rules come from here too
An application's eslint.config.ts is a spread, not a rule set, and it is the
whole of it:
import kiln from "@kilnjs/core/eslint";
export default kiln();Two lines, because there is nothing left to pass. tsconfigRootDir used to be
one: every application wrote __dirname so that lint meant the same thing run
from anywhere, and the default was process.cwd(), which did not. It is walked
up to the nearest tsconfig.json now.
An application that wants something differently puts it after the spread:
export default [...kiln(), { rules: { "max-lines": "off" } }];It stays a file in the application rather than something core generates, and
that is the one piece of toolchain configuration that does. An editor resolves
a config by walking up from the file it is linting, so a generated one under
.kiln/ would be found by kiln lint and by nothing a person is typing in.
What comes with it: the plugin set and type-aware parser, the rule set, the fourteen rules core ships itself, and the per-directory exemptions the framework's own layout needs - models are named after models, a model's methods are named by their receiver, seeds and queues may report progress, a controller is held to the shape core validates, a test may do what a test has to.
The fourteen are the ones no plugin ships. Six are about what generated code looks like when nobody is reading it:
| | |
| -------------------------------- | --------------------------------------------------------------------- |
| kiln/no-aphorisms | a comment is for the reason, not for a proverb |
| kiln/no-block-comment | /* */ hides how long it is. Autofixes to // |
| kiln/short-comments | two lines. Longer reasoning belongs in a README |
| kiln/no-em-dash | a dash nobody can type is a dash nobody can grep for |
| kiln/no-multiline-arrow | one line or a block, never a statement pretending to be an expression. An object literal is exempt: it is a value, and refusing it left one legal way to map a list to objects |
| kiln/descriptive-function-name | a verb, its object, and where a reader reads from |
There was a kiln/comment-density as well, a floor of a tenth of a file's
lines, and it is gone. A floor and a two line cap pull against each other: the
cap leaves less room per comment, so the floor starts asking for more of them,
and padding a file with small comments to clear a ratio is the opposite of the
point. A file with no comments is now a file with nothing worth saying.
One thing the ban took with it: knip reads a @public JSDoc tag to mean an
export is API even where nothing imports it, and a tag only counts in a block
comment. Four of those existed and two of them were the only evidence that
NAME and JWTPayload were meant to be reachable, so knip called them dead
the moment the comments became //. They are exported from their package's
barrel now, which says the same thing where the compiler and knip can both
read it. An export that is public is published; it does not say so in prose.
kiln/no-block-comment exempts an ESLint directive, and that is not a style
choice. A file-level eslint-disable is only honoured as a block comment:
written as // eslint-disable kiln/no-public it is silently ignored and the
rule goes on firing, so whoever followed the instruction is left with neither
the exemption nor a working comment.
It exempts a one-line @type or @satisfies annotation for the same
reason: it is the only way to type a plain .js or .mjs file, which
kiln.config.mjs is, and as // it says nothing to the compiler. The
exemption is one tag on one line, because prose with an @type buried in it
is still prose.
And eight are about endpoints, caches and errors, where forgetting to declare something is either a security decision or a promise nobody can check:
| | |
| ----------------------------- | ------------------------------------------------------------------- |
| kiln/no-public | reachable with no session, allowed per file with a stated reason |
| kiln/require-authorize | deny by default: state what the caller must hold |
| kiln/require-response | and what the endpoint answers with |
| kiln/serialisable-response | and that what it answers with is something JSON can carry |
| kiln/no-cross-site | a write accepting any origin while the session cookie rides along |
| kiln/no-returned-credential | a response promising a field whose name reads as a stored credential |
| kiln/endpoint-file-order | handler, method and path, permission, schema, default export |
| kiln/vault-is-the-only-export | a cache file exports its vault and nothing beside it |
And one is about neither, because a model method and a job raise errors too:
| | |
| ----------------------------- | ----------------------------------------------------------------- |
| kiln/no-request-in-context | a whole request, body or headers in an error's reported context |
The last three are declarations rather than settings, so each is allowed per
file with a reason, the way kiln/no-public is. A refusal that cannot be
answered is a refusal somebody turns off at the root.
kiln/descriptive-function-name is the one with options, because a verb
list is never finished:
"kiln/descriptive-function-name": ["error", {
verbs: ["ingest", "reconcile"], // on top of the ones core knows
ignore: ["handler", "queue"], // a name a contract chose, not its author
requireSource: true, // a reader that takes arguments says where from
minimumWords: 2, // a verb and the thing it acts on
}]An application passes its own words to kiln() rather than restating the
rule, because this config states that rule in three blocks with a different
ignore list in each and ESLint replaces a rule's options rather than
merging them: a restatement after the spread would silently drop whichever
block it did not also restate.
export default kiln({
vocabulary: { verbs: ["ingest"], ignore: ["mailman"] },
});It checks a standalone function and leaves a method alone, because a method
already has a subject: database.widget.prune() says what it prunes, while
prune() on its own could be anything. A module's own factory is exempt for
a related reason: metrics() in a config list names which module is on
rather than doing something. A package's factory is exempt by living in
src/module.ts; an application's own goes in vocabulary.ignore, which
reaches every block.
Four shapes fail it:
| | |
| ---------------------- | ---------------------------------------------- |
| routeOf(req) | the name is its return value, the verb is gone |
| repeatOf(entry) | the verb is there, what of pointed at is not |
| printed(names) | a participle, so it reads as a value |
| record(entry) | a verb with no object, so it could be anything |
What it asks for instead: getRouteFromRequest, buildRepeatFromSchedule,
buildPrintedLabels, recordWrite.
The second of those is the one worth knowing about, because the first pass
got it wrong. signIn and logOut end in a preposition and are missing
nothing, so a name that starts with a verb was exempted, which quietly let
scheduleOf, repeatOf and answerFrom through: schedule, repeat and
answer are all verbs. Only a particle can finish a phrasal verb, and of
is never one.
Its limit is a const that holds a function it was handed rather than one
written there: const carriedBy = buildMemoised(...) is not checked. A
type-aware version was tried and measured, and it flagged seventy locals and
loop variables whose values happen to be callable (const done = jest.fn(),
for (const endpoint of endpoints)) against three real finds. Nothing in the
syntax or the type separates defining a function from holding one.
A package lints itself with the same rules, through kilnPackage(), which
adds what keeps it publishable.
A module may ship a command
kiln <name> runs a command core does not have when @kilnjs/<name> exports
./cli, under tsx so it can read the application's own TypeScript. One rule
by shape rather than a list, so core still names no module: kiln permissions
is the permission module's own command, and core has never heard of it.
kiln check runs such a command as a step where one is installed, which is
how a module contributes a gate rather than a convention.
A module that writes code the application compiles against says so in its
manifest the same way, and kiln generate runs it after the database client:
"kiln": { "check": ["--check"], "generate": ["generate"] }That is kiln next --check as a gate and kiln next generate as a generator.
kiln setup runs kiln generate where it used to run kiln db generate
alone, so a fresh clone of an application with a front end gets its client
generated too, and create-kiln runs it last, before anything is linted.
setup, from nothing
kiln setup dev is meant to take a clean checkout to a running server, so the
order of its steps is the whole design: install, pins, install again, .env,
then the generated code, the hooks, the migrations and the seeds. Generating
the client used to come before .env was written, and Prisma's configuration
refuses to load without DATABASE_URL, so on a clean checkout it stopped
there every time with PrismaConfigEnvError.
Writing .env asks for nothing. It runs kiln env --env in the mode a
generator runs in, because that command loads app.config.ts and with it every
module's settings: on a clean checkout the variables it would validate are the
ones this step is about to write, so it refused to write them.
The migrations are applied, not authored. The step is migrate deploy,
which creates the database where there is none and applies what is committed.
migrate dev stops on a prompt for the name of a migration nobody asked for,
and a prompt inside a setup is a setup that never finishes; a schema that has
moved since its last migration is yours to name, and the comman
