@twzrd1111/agent-killswitch
v0.5.0
Published
Drop-in local reverse proxy that enforces hard token/cost budgets on OpenAI-compatible chat completions, with an instant file-based kill switch and an append-only JSONL spend ledger.
Maintainers
Readme
Agent Spend Kill-Switch
A drop-in local reverse proxy that sits in front of any OpenAI-compatible
/v1/chat/completions endpoint. It enforces hard token/cost budgets, gives
you an instant file-based kill switch, and writes every decision to an
append-only JSONL ledger.
Point any OpenAI client at it by changing one variable — OPENAI_BASE_URL —
and calls through the proxy are refused once they would cross the limits you
set. It caps a bill: it does not detect a loop, and by default a loop that
costs $0 (a free model) keeps running until you touch the kill file. The
opt-in request-rate limit caps calls a minute whatever they cost (see
"Breakers"). It listens on 127.0.0.1 by default and talks to one
configured upstream.
5-minute setup (npm / npx)
Requires Node.js 20 or newer. No global install needed:
# 1. One directory for the ledger and the kill file (same as the Docker flow)
mkdir -p ./data
# 2. Run — the only secret is your upstream key. With no config.yaml in
# the working directory the proxy uses the defaults from
# config.example.yaml (upstream https://api.openai.com/v1, port 4020,
# kill file ./data/.kill, ledger ./data/spend-ledger.jsonl).
OPENAI_API_KEY="$OPENAI_API_KEY" npx --yes @twzrd1111/agent-killswitch
# 2. Point any OpenAI client at it
export OPENAI_BASE_URL="http://127.0.0.1:4020/v1"To change limits or the upstream, write a config.yaml and pass it as the
first argument — npx --yes @twzrd1111/agent-killswitch ./config.yaml — or
set KILLSWITCH_CONFIG. config.example.yaml in this repository is a full
reference; every value has a KILLSWITCH_* env override (table below). The
kill file and the ledger land in ./data/ under the directory you run
from, so touch ./data/.kill stops it.
5-minute setup (Docker)
docker pull ghcr.io/twzrd-sol/agent-killswitch:0.2.1
mkdir -p ./data
docker run --rm \
-p 127.0.0.1:4020:4020 \
-v "$(pwd)/data:/app/data" \
-e OPENAI_API_KEY \
ghcr.io/twzrd-sol/agent-killswitch:0.2.1Then:
export OPENAI_BASE_URL=http://127.0.0.1:4020/v1Emergency stop, from the directory where Docker was launched:
touch ./data/.killA missing config file already uses the defaults, so you do not need to copy
config.yaml. The host ./data mount is only for the ledger and
touch ./data/.kill. The mounted directory preserves the ledger across
restarts. Workers sharing budgets must mount the same host directory.
Works with LangChain, AutoGen, CrewAI, Vercel AI SDK, Cursor, and anything
else that speaks /v1/chat/completions — streaming included.
The kill switch
# Stop all spend immediately (no upstream call is made after this)
touch ./data/.kill # or: curl -X POST http://127.0.0.1:4020/kill
# Resume (file-only; there is no HTTP reset endpoint)
rm ./data/.kill
rm -f ./data/.pauseWith the Docker setup above, ./data is mounted from the host, so from
the directory where Docker was launched, touch ./data/.kill reaches the
container.
Upgrading from 0.1.0 or 0.1.1: the default kill file moved from ./.kill
to ./data/.kill. The old config.example.yaml set file_path: "./.kill"
explicitly, so a config.yaml copied from it keeps the old path. In Docker
that path is /app/.kill, outside the mounted volume, and a host-side
touch never reaches it. Change it to ./data/.kill, then check with
curl http://127.0.0.1:4020/healthz ("killed": true after the touch).
A vetoed request returns an OpenAI-shaped error so frameworks handle it gracefully (HTTP 403 for the kill switch, 429 for budgets):
{"error": {"type": "killswitch_veto", "code": "budget_exceeded",
"message": "Hourly limit $5.00 hit", "spent_usd": 5.12, "limit_usd": 5.0}}If upstream does not answer within timeout_ms (default 60 s) the proxy
returns 504 upstream_timeout; if it cannot be reached at all, 502
upstream_unreachable (both "type": "killswitch_error"). Upstream may
still bill a request it received before the timeout, so the call keeps its
pre-call hold (below) and the ledger records that estimate. A stream that has
started is never cut off by the timeout.
In-flight holds
An allowed call holds its pre-call estimate from the moment it passes the check until it finishes, so concurrent calls see each other's spend:
- Prompt tokens are bounded by the size of the request: one token per byte of the body the proxy forwards, or of the body it received when that is larger (see "Prompt-token bound" below). That count is part of every token hold.
- With
max_tokens: the token hold ismax_tokensplus the request bytes. The dollar hold ismax_tokensat the model's output rate plus the request bytes at its input rate. - Without
max_tokens: the token hold is the request bytes. The dollar estimate is the per-request cap (max_cost_per_request_usd, $0.50 by default), or the cost of the request bytes at the input rate when that is higher, and the hourly and daily checks run against it. - A prompt that is not plain text (an image, audio or file part, tools,
a response schema, any request field the proxy does not know): the dollar
hold is never below the per-request cap, with or without
max_tokens. - A model priced at $0 holds no dollars. Its token hold is the same as any other model's.
When the call ends, the hold becomes a ledger line:
- Upstream reports usage: the real cost is written (
"metered": "usage"). - The call finished without usage: the held estimate is written
(
"metered": "held_estimate"). This covers a 2xx with no usage, a stream that ends without a usage event, a timeout, an unreachable upstream, or an error after the request was sent. A restart replays it. - Upstream answers non-2xx without usage (for example a 402 payment challenge): the hold is released and no spend is written.
A veto before the request is sent leaves no hold. Holds live in memory only:
a crash while a call is in flight drops its hold rather than charging an
unsettled estimate forever. /healthz reports in_flight.
What the kill switch protects against
It caps what a runaway agent can spend (a loop, a retry storm, a prompt
that will not end) and refuses the next call once a limit would be crossed.
By default it does not stop the loop itself: calls that stay under the limits
keep going until you touch the kill file, and a call already sent upstream
runs to its end. Two opt-in breakers (below) pause on a loop's shape instead
of its spend: a run of upstream failures, or more calls a minute than you
allow. It does not stop an adversarial agent either. Any local process
that can write the sentinel directory can lift the kill by removing .kill
or .pause, and that includes the agent itself. HTTP has no reset endpoint.
If the agent could act against you:
- run the proxy as a different user or in a container, with
data/not writable by the agent; - set
KILLSWITCH_KILL_API=falseso HTTP/killis off (the file still works); - keep the upstream key out of the agent's environment, or it can call upstream directly and skip the proxy. Traffic that never reaches the proxy's port is not capped.
Breakers (opt-in)
Both breakers are off by default. Each writes .pause, never .kill, so a
trip also halts the sibling payment gate when it shares the state directory,
and each logs its trip on the hash-chained ledger.
Upstream-failure breaker (KILLSWITCH_UPSTREAM_FAILURE_THRESHOLD,
KILLSWITCH_UPSTREAM_FAILURE_COOLDOWN_MS). It counts consecutive upstream
failures:
- a timeout (504), an unreachable upstream (502), or an error after the request was sent;
- any 5xx answer;
- any non-2xx answer to a call that carried a payment credential.
An unpaid 402 challenge, 400 or 429 neither counts nor resets the count: the
upstream is up and answering. A 2xx resets it. At the threshold the breaker
trips. The call that tripped it gets its own answer, .pause is written for
the cooldown (default 60 s, at most 24 h), and calls are refused with 429
payment_gate_halt until the pause ends. Then the breaker is half-open:
calls flow again, and the first to finish with a 2xx or a failure decides.
A failure re-trips it at once; a 2xx closes it and resets the count.
Request-rate limit (KILLSWITCH_MAX_REQUESTS_MINUTE). It caps how many
calls the proxy forwards in any 60 s, whatever they cost. The call over the
limit is refused with 429 request_rate_limit and writes .pause for 60 s.
It counts calls, whatever is in them: it is a rate limit, not loop
detection.
The token limit does not do this job. At the default 100,000 tokens a minute, a call of about 100 bytes holds about 100 tokens while it is in flight, and once answered it counts only the tokens the upstream reports, so thousands of such calls a minute pass. A call answered non-2xx without usage releases its hold and records nothing, so a loop of failing calls is never counted at all.
Both breakers count chat completions and MPP topUps, the calls a pause
refuses. A close, and a bodiless call without a credential, are not
counted and still pass a pause. Counting is per process: two proxies on one
state directory share the .pause, not their counts.
- An upstream-failure trip writes
.pauseas the tripping call ends, just after its answer is sent. The proxy takes no other call in between, but a process that reads the file the instant that answer arrives can still see the old state for a few milliseconds. A request-rate trip writes.pausebefore its refusal is sent. - A trip never shortens a longer pause already in force. It never replaces
a corrupt
.pause, which halts until a person removes it. - If
.pausecannot be written (a read-only state directory), this process holds the pause in memory and refuses until it ends./healthzthen reports"pause_held_in_memory": true. That pause does not reach the payment gate. - Removing
.pauseby hand lifts a breaker's pause like any other. The upstream-failure breaker is then half-open, and the request-rate window still counts the calls of the last 60 s.
/healthz carries a breakers object while either breaker is on. state
is closed, open (tripped, pause in force) or half_open:
"breakers": {
"upstream_failure": {"state": "closed", "consecutive_failures": 0, "threshold": 5, "cooldown_ms": 60000},
"request_rate": {"requests_in_window": 12, "limit": 120, "window_ms": 60000}
}An upstream-failure trip appends a line of its own: model (breaker),
allowed: false, cost 0, veto_reason: "upstream_failure_breaker", and the
request_id of the call that tripped it. A request-rate trip is the veto
line of the call it refused. Both carry a breaker object:
{"name": "upstream_failure", "event": "trip",
"reason": "5 consecutive upstream failures; the last was upstream_status_503",
"pause_until_unix_ms": 1794830400000, "consecutive_failures": 5, "threshold": 5, "cooldown_ms": 60000}event is retrip for a failure while half-open. pause_until_unix_ms is
null when a longer pause was already in force. pause_write_failed: true
marks a pause held in memory. A request-rate record carries
requests_in_window, limit and window_ms instead. Its refusal body
carries the request count and the limit in spent_usd and limit_usd, as
the token-per-minute refusal carries tokens there.
Configuration
YAML file plus env overrides (env wins). A missing config file uses the
defaults below. Copy config.example.yaml to config.yaml only if you
want to edit the limits in a file. kill_switch.enable_api_endpoint must be
a literal YAML boolean; strings such as "false", off, and no are rejected
at startup rather than coerced:
| Env var | YAML key | Default |
|---|---|---|
| KILLSWITCH_CONFIG | — (config path) | ./config.yaml |
| KILLSWITCH_UPSTREAM_BASE_URL | upstream.base_url | https://api.openai.com/v1 |
| KILLSWITCH_API_KEY_ENV | upstream.api_key_env | OPENAI_API_KEY |
| KILLSWITCH_UPSTREAM_TIMEOUT_MS | upstream.timeout_ms | 60000 (0 disables) |
| KILLSWITCH_MAX_TOKENS_MINUTE | limits.max_tokens_per_minute | 100000 |
| KILLSWITCH_MAX_USD_HOUR | limits.max_usd_per_hour | 5.00 |
| KILLSWITCH_MAX_USD_DAY | limits.max_usd_per_day | 25.00 |
| KILLSWITCH_MAX_COST_REQUEST | limits.max_cost_per_request_usd | 0.50 |
| KILLSWITCH_MAX_REQUESTS_MINUTE | limits.max_requests_per_minute | 0 (off) |
| KILLSWITCH_UPSTREAM_FAILURE_THRESHOLD | upstream.failure_threshold | 0 (off) |
| KILLSWITCH_UPSTREAM_FAILURE_COOLDOWN_MS | upstream.failure_cooldown_ms | 60000 (1 to 86400000) |
| KILLSWITCH_KILL_FILE | kill_switch.file_path | ./data/.kill |
| KILLSWITCH_KILL_API | kill_switch.enable_api_endpoint | true |
| KILLSWITCH_LEDGER_PATH | logging.path | ./data/spend-ledger.jsonl |
| KILLSWITCH_PORT / PORT | server.port | 4020 |
| KILLSWITCH_HOST | — (bind address) | 127.0.0.1 (Docker image sets 0.0.0.0) |
| KILLSWITCH_FREE_MODELS | — (comma-separated exact model ids) | unset |
The three breaker settings take whole numbers only: a YAML number (not a
quoted string, a decimal or a boolean), or ASCII digits in the env var.
Anything else is refused at startup. In a twzrd.binding_policy.v1 file
the keys are compute.max_requests_per_minute,
compute.upstream_failure_threshold and
compute.upstream_failure_cooldown_ms.
When KILLSWITCH_FREE_MODELS is set, a model outside that list is rejected
with 403 model_not_allowed and never reaches upstream. The proxy also
skips the upstream Authorization header and does not read the API key
environment variable. Leave it unset for the normal keyed OpenAI proxy.
A listed model that has no row in the price table is priced at $0: the
list is your statement that it costs nothing, which is what a local model
behind Ollama or a free catalog entry is. The dollar limits then never
trip for it; the per-minute token limit and the kill file still apply. A
listed model that does have a price row (gpt-4o, for example) keeps that
price, and a dated or suffixed id of a priced model (gpt-4o-2024-11-20)
keeps the $15/$60 fallback, or that row's price where it is higher.
The $0 price is for unpaid calls only. Keyless mode is also how an
MPP/x402 buyer runs the proxy, so a call that carries a payment credential
(X-Payment, Payment-Signature, any X-Payment-*, or Authorization:
Payment ...) is priced as if the model were not on the list, and the
dollar limits apply to it. The same goes for a model the built-in table
lists at $0 (nvidia/nemotron-3.5-bolt): a paid call to it is priced at
the $15/$60 fallback. A rate you set yourself in model_rates is used as
given, at $0 too. Do not list a model your upstream bills for in a way the
proxy cannot see.
A $0 call holds no dollars. Without max_tokens its token hold is its
request bytes, so a burst of small parallel calls to a local model all
reach upstream, and the per-minute token limit counts what the responses
report once they land. To cap the number of calls, set the request-rate
limit (see "Breakers").
Every veto, and every allowed call except one answered non-2xx without
usage, appends one line to the JSONL ledger with timestamp, model, token
counts, cost_usd, allowed, veto_reason, and request_id; allowed
lines also carry metered (usage or held_estimate). An allowed line is
written when the call ends. A breaker trip writes a breaker record (see
"Breakers"). Restart replays the last 48 h to rebuild the spend
windows; the ledger file is the source of truth. Each line also carries
prev, which links it to the line before it (see "Ledger integrity").
Ledger integrity
Every line the proxy writes carries prev: the SHA-256 (hex) of the exact
bytes of the last complete line before it, or "genesis" for the first line.
Editing, removing or reordering a line breaks the link of the line after it.
twzrd-x402-gate's durable spend ledger uses the same convention.
Check a ledger offline (no config, API key or network needed):
npx @twzrd1111/agent-killswitch verify-ledger ./data/spend-ledger.jsonl
# in the container:
docker exec <container> node dist/index.js verify-ledgerWithout a path it reads KILLSWITCH_LEDGER_PATH, else
$TWZRD_STATE_DIR/spend-ledger.jsonl, else ./data/spend-ledger.jsonl. If
your config sets logging.path, compute.ledger_path or state_dir, pass
the ledger path.
Exit 0 means the chain is intact, 1 means it is broken (or --strict found
something), and 2 means the file could not be checked. --json prints the
full report.
What the chain does not prove on its own:
- The newest line. Nothing commits to it yet. Removing it, or rewriting
every line and recomputing every hash, still verifies. The command prints
the
headhash. Record it somewhere the ledger's writer cannot change: a later run that shows a different head over the same lines means the history was rewritten. - Several copies on one file. Each copy reads the last line just before it
appends, so the chain stays linear. Two copies appending at the same moment
both name the same
prev. The command reports that as a fork, and one row of the fork is then committed to by no later row.--strictrefuses forks, unparseable lines and rows withoutprev, for one copy per file. - Older rows. Rows written before this version have no
prev. They are reported and not checked; the first chained row commits only to the last of them.
Programmatic admission (createBudgetGate)
For in-process agents — for example Mastra's beforeToolCall hook
(@mastra/core 1.49.0 or later) — the package also exports the same pure
gate math the proxy uses. No server, no ledger, no I/O:
import { budgetInputForChatRequest, createBudgetGate } from "@twzrd1111/agent-killswitch";
const gate = createBudgetGate({
maxTokensPerMinute: 100_000,
maxUsdPerHour: 0.5,
maxUsdPerDay: 2,
maxCostPerRequestUsd: 0.04,
// optional, same semantics as the proxy:
// modelRates, allowedModels, zeroPriceModels
});
// Inside beforeToolCall (it runs before a tool call, not a model call):
// `text` is the exact chat-completions body you are about to send.
const text = JSON.stringify(body);
const request = budgetInputForChatRequest(text); // model, maxTokens, promptBytes, promptTextOnly
const decision = gate.evaluate({
...request,
window: { minuteTokens, hourUsd, dayUsd }, // you own these, holds included
});
if (!decision.allowed) {
// decision.status (429/403), decision.code, decision.message
return;
}
// Synchronously, before the first await, add the hold this decision assumed.
// Mastra runs tool calls in parallel; two calls that read the same unheld
// window both pass.
const est = gate.estimate(request);
minuteTokens += est.tokens;
hourUsd += est.usd;
dayUsd += est.usd;
// Send `text` unchanged. After the call completes, replace the hold with
// the actual cost (subtract est, add what was actually used), the same
// settle the proxy does.Add the hold synchronously, before the first await. estimate()
returns the same token/dollar hold the proxy uses while a request is in
flight; two parallel calls must each see the other's hold in the window.
Read the request with budgetInputForChatRequest(text). Give it the
exact string you hand to your HTTP client. It returns the model, the output
limit and the text-only flag read with the same functions the proxy uses,
and the UTF-8 byte count of that text. It checks the parsed text rather
than the object you built, so a toJSON or class instance cannot show the
check one shape and send another. Send text without duplicate keys
(JSON.stringify output has none): JSON.parse keeps the last of a
duplicated key, and an upstream that keeps the first would bill outside
this reading. The proxy's own byte count for the same text can be higher,
never lower, because it re-serializes the body before forwarding (for
example, it adds stream_options.include_usage to a streaming request). It
throws on text that is not a JSON object with a non-blank model.
Building the input by hand is supported but easy to get wrong:
maxTokensischatOutputLimit(body): the larger ofmax_tokensandmax_completion_tokens, timesn. A value below 1 is no limit: passnull, which holds the per-request cap. Passing0holds no output at all, while an upstream may read 0 as "no limit". When the product overflows,chatOutputLimitreturnsInfinity, whichevaluate()andestimate()accept as unbounded output and every limit refuses.promptBytesis the UTF-8 byte length of the string you send, not of an earlier copy of the body.promptTextOnlyisisTextOnlyPrompt(JSON.parse(text)). Among other things it is false whenmax_completion_tokensis set without a usablemax_tokens, or whennorservice_tierbill above one standard-rate completion.
Leave promptBytes out and the dollar estimate is never below
maxCostPerRequestUsd, the same as a call without maxTokens. Leave
promptTextOnly out, or pass false, and the bytes still count but the
dollar estimate is again never below the cap. A promptBytes that is not a
non-negative whole number, or a promptTextOnly that is not a boolean,
throws.
Paid calls. A call that carries an x402 or MPP credential must pass
zeroPriceModels: [] for that request — that is what the proxy does. A
credential opts the call out of free-model pricing. zeroPriceModels: []
does not throw; it means "no free model for this request": neither a
model on the gate-wide list nor one the built-in table lists at $0
(nvidia/nemotron-3.5-bolt) is priced at $0. A modelRates entry you set
to $0 still applies. For an unpaid call with no free list, leave
zeroPriceModels out or pass null; [] there prices the built-in $0
model at the fallback.
modelRates shape. modelRates takes { inputPerMTok, outputPerMTok }
per model id, with an optional perRequestUsd (a flat USD amount added to
each request's estimate and cost). This is not the shape of the config file's model_rates,
which uses in and out (see "Operator model-rate overrides" below); the
config shape is rejected here:
createBudgetGate({
/* ...limits, */
modelRates: { "my-model": { inputPerMTok: 1, outputPerMTok: 2 } },
});{ in, out } is not accepted and throws. The config and environment
overrides (model_rates / KILLSWITCH_MODEL_RATES) keep the in/out
shape they have always used; only this programmatic option differs.
Allowlist. allowedModels: [] throws at construction — pass null (or
omit it) for "no allowlist". Reading [] as "no allowlist" would silently
open the gate, so the empty list is rejected instead.
Limits, model rates and model lists are snapshotted and frozen when the
gate is constructed (and each request field is read exactly once), so a
caller mutating its own objects or accessors afterwards cannot loosen a
live gate. Malformed options, windows, requests and model rates throw a
TypeError instead of failing open.
gate.estimate({ model, maxTokens?, promptBytes?, promptTextOnly? })
returns the same token/dollar hold the proxy uses while a request is in
flight; use it to add your own in-flight holds to the window. The gate
does not record spend — after an allowed call completes, add the actual
cost to your window yourself. For the ledger, the kill file and the
upstream lifecycle, run the proxy.
Paid tool calls (createToolGuard)
createBudgetGate covers chat completions. createToolGuard covers the
other things an agent pays for: a search API, a scraper, an x402 or MPP
resource. It uses the proxy's three files (it reads .kill, and reads and
writes the ledger and .pause), so point it at the proxy's state directory
and tool spend and inference spend share one hourly and one daily budget,
one kill file and one hash-chained ledger.
import { createToolGuard, ToolNotSentError, ToolRefusedError, ToolSettleError } from "@twzrd1111/agent-killswitch";
const guard = createToolGuard({
ledgerPath: "/var/lib/killswitch/ledger.jsonl",
killFilePath: "/var/lib/killswitch/.kill",
pauseFilePath: "/var/lib/killswitch/.pause",
allowedTools: ["search", "news"], // required; there is no "allow all"
maxCostPerCallUsd: 0.002,
maxUsdPerHour: 0.05,
maxUsdPerDay: 0.05,
});
try {
const { result } = await guard.run(
// The most this call can cost, from the tool's published price and the
// request you are about to send.
{ tool: "search", worstCaseUsd: 0.0005, requestId },
async () => {
if (!query) throw new ToolNotSentError("empty query"); // nothing left the process
if (query.length > 400) throw new RangeError("query too long"); // raised here, before the client is called
const response = await search(query);
return { result: response.results, billedUsd: response.billedUsd };
},
// Optional: true only for an error raised before any byte of the request
// could leave the process. Never decide it from a network error code.
{ isNotSent: (err) => err instanceof RangeError && err.message === "query too long" },
);
} catch (err) {
if (err instanceof ToolRefusedError) {
// err.code says why. Nothing was sent and nothing was spent.
} else throw err;
}Before the call the guard refuses, and fn never runs, when:
| Code | Reason |
| --- | --- |
| kill_switch | the kill file exists |
| payment_gate_halt | a .pause is in force, or the sentinel is corrupt |
| tool_not_allowed | the tool is not on allowedTools |
| unbounded_cost | worstCaseUsd is not a finite non-negative number |
| per_request_cap | the worst case is over maxCostPerCallUsd |
| budget_exceeded | the worst case would take the hour or the day over its limit, or an allowed line in the last 24 hours has a cost that is not a finite non-negative number |
An allowed call holds its worst case until it settles, so parallel calls
cannot both pass on the same unspent budget. After the call the guard
records what the tool billed. It trips when the bill is above the worst
case (the full bill is still recorded) or when there is no readable bill:
fn threw, or billedUsd was not a finite non-negative number. The worst
case is then recorded as spend. A trip writes .pause for tripPauseMs
(default one hour, from one second to one day), so the proxy and the
payment gate on the same state directory stop too. It never shortens a
longer pause and never writes .kill. The pause ends on its own when the
time is up; nobody has to clear it. To resume early, delete .pause.
run() treats every error from your function as a call that may have
been billed, with one escape for a call that was never sent: throw a
ToolNotSentError, or pass isNotSent and return exactly true for
errors your client raises before it sends anything. The hold is then
released, nothing is recorded and nothing is paused. The classifier must
be true only when no byte of the request can have left the process: a
refusal your own code raised before it called the client, or a
connect-phase error from a client with redirects turned off. A DNS error
(ENOTFOUND) or a refused connection from a client that follows redirects
does not qualify: the first request can reach the tool and be billed
before a later hop fails with the same code. A timeout or a dropped
connection is not "never sent" either. The
escape covers only what fn throws: once fn has resolved, the call
counts as sent, whatever its result looks like.
If the ledger cannot be written when a call settles, settle() throws,
the guard still pauses (on disk if it can, and in memory for every guard
in the process on that ledger file), the spend keeps counting in memory at
the larger of the bill and the worst case, and the hold stays open so you
can call settle() again. The bill is fixed at that first attempt: a
retry records it whatever you pass, and the hold can no longer be
released. From run() this arrives as a ToolSettleError carrying the
tool's result and the hold, so a result you paid for is not lost:
try {
const { result } = await guard.run(call, fn);
} catch (err) {
if (err instanceof ToolSettleError) {
use(err.result);
// later, once the ledger is writable again:
err.hold.settle();
}
}Before each line it writes, the guard ends a torn last line (a write cut short by a full disk or a crash), so the new line is not glued to the fragment and lost.
guard.authorize(call) is the synchronous form: it returns the decision
and, when allowed, a hold with settle(billedUsd) and release(). Call
it before the first await of the tool call. Use release() only for a
call that was never sent. guard.spend() returns the hour and day totals,
holds included.
Every refusal, settled call and trip goes on the ledger with model
"(tool)" and a tool
object (name, event of settled, refused or trip, worst_case_usd,
and billed_usd on a settled line). verify-ledger checks these lines
like any other.
Limits of the guard:
- It trusts the caller's
worstCaseUsdand the tool's own bill. It cannot see a charge the tool does not report. - A hold lives in the process that made it. A crash between
authorizeandsettledrops the hold and records nothing, as with the proxy. - An open hold counts in full until it is settled or released, however long the call takes. A hold you never close keeps its budget until the process exits.
- Guards in one process that use the same ledger file (by its real path, so through a symlink too) share their holds and any pause held in memory. The proxy's own holds, and holds in another process, are not visible: those callers see each other's settled spend only.
- The ledger's sliding windows use the wall clock. A clock that steps backwards hides spend stamped after the new time until the clock passes it again.
- A refusal for a budget does not write
.pause; only a trip does. - Dollar comparisons allow 0.000000001 USD of slack so that binary rounding of prices such as 0.0005 cannot refuse or trip on its own.
Tests
npm ci && npm testOffline: a stub upstream on 127.0.0.1 stands in for OpenAI, so no key,
network or spend is needed. Covered: the default kill file (403, upstream
never called), the kill API, ledger replay over the hourly limit (429),
replay skipping vetoed, stale and torn lines, metered cost on a normal and
a streamed request surviving a restart, the 504/502 upstream failures
keeping the held estimate, overlapping calls seeing each other's holds
(dollars and tokens), the per-request cap standing in for a missing
max_tokens, the prompt-token bound (two seeded 25,000-case differential
runs against the 0.1.10 gate: the estimate never falls, no request that was
refused is allowed,
and the proxy and createBudgetGate return the same estimate for the same
request), a 200 and a stream without usage keeping the hold, an
upstream HTTP 402 whose WWW-Authenticate / PAYMENT-REQUIRED challenge
reaches the client, a paid retry whose X-Payment / Payment-Signature /
X-Payment-Signature headers reach upstream, and Payment-Receipt /
X-Payment-* on 200 (stream and non-stream). The ledger chain: rows written
through the proxy (allowed and vetoed) verify with --strict; an edited,
removed, swapped or garbled line is reported at the line after it; forks
from two copies (refused with --strict), older rows, torn tails and a
ledger written by twzrd-x402-gate verify; and the verify-ledger exit
codes. A test also states the limit: dropping the newest line or rewriting
every hash verifies, and only a recorded head shows it. The programmatic
export: the . export resolves to dist/api.js with its declarations, the
decisions are identical to the proxy's on the same input, and malformed
options, windows, requests and model rates throw rather than fail open. MPP
sessions and flat rates: a close or topUp with no body passes without a
hold (credential forwarded, body dropped, receipt returned, one cost-0
session_action line); a bodiless voucher, open or x402 header is refused
before upstream; the kill file refuses all of them and a pause refuses a
topUp; a flat per_request row holds and ledgers exactly its amount, so
an open, four vouchers and a close ledger 3,750 base units on a chain that
verifies with --strict. Halts and breakers: each spend cap (tokens per
minute, dollars per hour, dollars per day) refuses before upstream and
writes .pause for its own window; both breakers are off by default; the
upstream-failure breaker trips at its threshold, refuses while paused, goes
half-open, re-trips on one failure and closes on one 2xx; timeouts, dropped
connections and paid non-2xx answers count, unpaid 402/400/429 do not; a
topUp counts and a close does not; the request-rate limit refuses the
call over the limit; a trip does not shorten a longer pause or replace a
corrupt one; a pause that cannot be written is held in memory; strict
config parsing. A cross-reader test runs the real payment gate against a
loopback seller: a breaker's .pause in a shared TWZRD_STATE_DIR refuses
the paid call and the board fallback before anything is paid. Another
passes a signed V7 receipt through the proxy in Payment-Receipt: the
bytes arrive unchanged and still verify, and a changed one fails the
gate's verifier (the proxy does not verify receipts).
Pricing notes
Costs use a static per-model table (src/pricing.ts, checked against
provider catalogs; sources below). There are no live pricing fetches by design.
An unknown model is estimated at a fixed fallback of
$15 input / $60 output per million tokens rather than rejected —
failing closed on cost beats failing open. A model id that is not in the
table is priced at that fallback (or at a dearer row it extends; see "Built-in
price rows") unless you put it on KILLSWITCH_FREE_MODELS, which
prices it at $0 (above); nvidia/nemotron-3.5-bolt is listed at $0 for
unpaid calls. The ledger shows exactly what was assumed.
Prompt-token bound
The checks before a call count prompt tokens without a tokenizer: every
byte of the request body counts as one prompt token, priced at the model's
input rate. A 300 kB text prompt on gpt-4o with max_tokens: 16 is
estimated at 300,000 prompt tokens and $0.75. With the defaults it is
refused by the 100,000 tokens-per-minute limit, which is checked first; with
that limit raised it is refused by the $0.50 per-request cap. Through 0.1.10
it was forwarded.
Why bytes are an upper bound for text:
- A tokenizer splits text into pieces, and every piece covers at least one byte of the text. So text of N bytes is at most N tokens. This holds for BPE, WordPiece, unigram and character tokenizers alike, and it is the reason no tokenizer has to be named.
- A JSON string is never shorter than the UTF-8 text it decodes to: a
character written as itself has the same bytes, and every escape
(
\n,\u00e9, a surrogate pair) is longer than what it stands for. The strings in a request do not overlap, so all the text in a body of B bytes is at most B bytes, and at most B tokens. - The bytes counted are the bytes the upstream receives. The proxy parses
the request and forwards its own serialization, which can be longer than
what arrived: a byte that is not valid UTF-8 becomes a 3-byte
replacement character, and a number such as
1e20is written out in full. So the proxy counts the text it forwards, or the size it received when that is larger.
What the argument does not prove, and what the proxy does about it:
- Framing tokens. An upstream adds a few tokens per message (role
markers, separators, the reply primer). The bound relies on the JSON
around each message (at least
{"role":""}, 11 bytes) being at least as large. That is an assumption about the upstream's chat template, not a proof. - Content that is not text in the body. An image, audio or file part,
a file or URL the upstream fetches, tools, a response schema, a search
plugin: the upstream can bill prompt tokens for these that no byte count
bounds. For such a request the dollar estimate is never below the
per-request cap, the same stand-in a missing
max_tokensgets. The test is an allowlist (isTextOnlyPrompt): plain-text messages plus sampling and transport fields pass, and any field the proxy does not know counts as not plain text. Message fields are checked by shape as well as by name:nameandtool_call_idmust be strings, andtool_callsandfunction_callmay hold only string ids, names and argument strings. - Model variants. A model id with a
:suffixcan make the upstream add prompt content of its own (a web-search variant, for example). Only the suffixes:free,:nitro,:floor,:extendedand:thinkingkeep the text-only bound, and every suffix on the id must be one of them (:online:freedoes not pass). Any other suffix, including a local tag such asqwen3:14b, and any id with an@(a preset), holds at least the per-request cap; for a model priced at $0 that changes nothing, because it holds no dollars. A model that searches or routes with no marker in its id (a search model, an auto-router) is not detected: its added tokens are the next item. - Billing options.
nother than 1 and aservice_tierother thandefaultorflexhold at least the per-request cap, because they change what the call bills without adding a byte. The price rows for OpenAI models are standard-tier prices, so aprioritycall is still ledgered at the standard rate when upstream reports usage. - Tokens the upstream adds on its own, such as a hidden system prompt. No request-side check can see those.
The bound is deliberately loose. English text is several bytes per token,
so the estimate of a text prompt is several times its real size, and inline
base64 media counts byte for byte. If a large request you want is refused,
raise max_tokens_per_minute or max_cost_per_request_usd; the hold is
replaced by the real cost as soon as upstream reports usage.
A request whose own count is over max_tokens_per_minute is refused with
budget_exceeded and no pause file is written, because waiting cannot make
it fit. With the default limit of 100,000 that is any request over about
100 kB, which a long plain-text history can reach: set the limit to at
least the largest body you expect, in bytes, plus its max_tokens.
The output side of the estimate is the larger of max_tokens and
max_completion_tokens, multiplied by n when the body asks for several
completions. A limit below 1 counts as no limit. When
max_completion_tokens is the only limit, the dollar hold is never below
the per-request cap, because an upstream that does not know the field
would generate without one.
While the hour or day spend is at or over its limit, every request that passes the kill switch, a non-budget pause and the model allowlist is refused with the daily or hourly refusal and its long pause file (the daily one when both are spent), before the per-minute and per-request checks, however large the request is. When the budget still has room:
- A request too large to pass on its own (over the per-minute limit or the per-request cap) is refused without a pause. That includes some requests for which 0.1.10 wrote the hourly or daily pause because its smaller estimate did not fit what was left.
- A request whose estimate does not fit what is left of the hour or day gets the hourly or daily refusal and pause, as in 0.1.10. Because the estimate is larger than in 0.1.10, this can now happen where 0.1.10 forwarded the request.
The proxy does not shorten a pause it saw in force when it checked the request: a 60-second per-minute refusal leaves an hourly or daily pause file as it is, and an hourly refusal leaves a daily one. A pause that another process writes at the same moment can still be replaced.
A call that is not plain text (one that sends tools, for example) holds
at least the per-request cap. If upstream answers it without usage, the
ledger records that hold, so with the defaults ten such calls use the $5
hourly limit. Upstreams that report usage are ledgered at the real cost.
MPP sessions
An MPP session client (mppx tempo.session) pays with a signed voucher on
each request and manages its payment channel with two more calls: topUp
(add deposit) and close (settle and end the channel). mppx sends both as a
POST with no body to the URL of the last paid request, which behind this
proxy is /v1/chat/completions. The proxy reads the action in the
Authorization: Payment credential (it does not verify the signature; the
seller does) and:
- forwards a
closeortopUpupstream with no body and no budget hold: it runs no inference. Any body sent with one is dropped. The seller's status,Payment-Receiptand challenge headers come back as sent. The ledger gets one line at cost 0 withsession_actionset andmodel"(session-management)". - forwards a request with no body and no payment credential the same way
(
session_action: "empty"); the seller answers it with a 402 challenge or an error. - refuses (400, never forwarded) a request with no body that carries any
other credential: a voucher, an open, an x402 header or an unreadable
Paymentvalue. A seller may accept a voucher on an empty request, and that would move money this proxy never metered.
The kill file refuses all of them (403 kill_switch). A pause refuses a
topUp (429 payment_gate_halt) and lets a close through, because a
close ends the channel and returns the unspent deposit. The hourly and
daily dollar limits do not apply to either, but the pause a budget refusal
writes still refuses a topUp. The proxy does not read voucher amounts: a
close settles whatever the client's wallet signed. It meters the requests
that pass through it, so give each request the seller's price with a flat
per_request rate (see "Operator model-rate overrides"); for a seller that
takes 750 base units of a 6-decimal stablecoin per request, that is
{in: 0, out: 0, per_request: 0.00075}, and five requests ledger $0.00375
(3,750 base units).
Upstream payment headers an MPP/x402 buyer needs — WWW-Authenticate
challenges, PAYMENT-REQUIRED, Payment-Receipt, and X-Payment-* — are
forwarded to the client on both stream and non-stream responses, including
HTTP 402. On the retry, client X-Payment, Payment-Signature,
X-Payment-Signature, and other X-Payment-* headers are copied upstream.
The proxy still injects Authorization: Bearer ${OPENAI_API_KEY} by default
and passes through the client's Authorization only when it is an explicit
402 payment credential (Payment scheme, or a non-Bearer value with a
payment header) — never as a second API key. Hop-by-hop headers,
Set-Cookie, and other secrets are not copied.
What it is not (v0.1 scope)
OpenAI chat completions only. No Anthropic, no embeddings/images, no tokenizer, no multi-upstream or key rotation, no dashboard, no Redis/SQLite.
Paid support
This is a local binary with no SaaS backend. Annual support (priority fixes,
custom limits/policies, audit-export help) is available. Open an issue at
https://github.com/twzrd-sol/twzrd-trust/issues with agent-killswitch in
the title, your use case and team size. Bugs go to the same tracker.
Served-model telemetry
New ledger rows keep model as the requested ID for compatibility and add
requested_model, served_model (string or null), and model_mismatch.
The proxy reads the JSON response's model, or the last valid model from
SSE data events, independently of whether usage is present. Missing, invalid,
blank or over-512-character IDs are recorded as null. A mismatch is exact string
inequality; aliases and dated IDs can therefore flag. False with a null served
model means unknown, not a verified match. Vetoes have no served model.
These are upstream-reported IDs, not execution attestations. Detection is telemetry; it does not reject responses or change billing to a possibly untrusted returned ID. Cost accounting continues to use the requested model. If a stream aborts before forwarding completes, existing hold settlement applies and the served ID may be unknown. No prompts, completion text, or response bodies are added to the ledger.
Operator model-rate overrides
Rates are USD per one million tokens, with separate in and out values:
export KILLSWITCH_MODEL_RATES='{"deepseek-reasoner":{"in":0.55,"out":2.19}}'An entry may add per_request: a flat USD amount charged on every request on
top of the token rates. A seller that bills one fixed price per request (an
MPP session voucher, a per-call x402 price) is a row with no token rates:
export KILLSWITCH_MODEL_RATES='{"infer-mpp/twzrd-coder":{"in":0,"out":0,"per_request":0.00075}}'Such a row holds exactly per_request before the call and ledgers exactly
per_request after it, with or without upstream usage, at any max_tokens
or prompt size: no token count can change what it costs, so the
per-request-cap floor that stands in for an unbounded output does not apply.
Tokens still count toward the per-minute token limit. A row with token rates
and per_request adds the two, and the floor applies as before. A row is
free only when in, out and per_request are all 0.
This is a configuration example, not a current DeepSeek price claim. In the legacy
config.yaml, the equivalent is:
model_rates:
deepseek-reasoner: {in: 0.55, out: 2.19}Exact ID precedence: environment entry > config entry > built-in table > existing
unpaid free-model allowlist rule > fixed $15/$60 unknown fallback, raised to
any dearer row or override the id extends with - or :. A built-in $0
row belongs to the unpaid free-model rule: a call that carries a payment
credential skips it. Each override
requires both finite nonnegative numeric fields (per_request, when present,
must be one too); malformed JSON, missing fields, negative values and unknown
fields fail startup. Zero rates are an explicit
operator assertion. Request bodies cannot override these rates. Restart to reload.
The shared binding-policy schema is unchanged; use the environment override with it.
The same rate lookup drives admission, in-flight holds and usage-based settlement. Existing ledger amounts are retained on restart. Adding a high-priced override does not raise the fallback for unrelated unknown IDs, only for IDs that extend it. Cached tokens, audio, tools, provider-specific fees and route changes can make actual provider charges differ.
Built-in price rows
Checked 2026-11-10 (OpenAI rows 2026-11-07). Each row is the highest price the
upstream can bill for that exact id, including any premium the request body can
select, because the agent being limited writes the body and the proxy forwards
it. Rows use the top prompt-length tier and already include announced price
increases. A row is a conservative budget estimate for the limits, not the price
a given call pays: under-pricing is the unsafe direction, so a row usually
overstates a call. If your agents cannot reach a premium (you pin one route, or the upstream
ignores the field), set an override (above) at the price you actually pay. The
full list, grouped by source, is in src/pricing.ts.
| Group | Ids | Source and rule |
|---|---:|---|
| OpenRouter (anthropic/…, deepseek/…, google/…) | 37 | https://openrouter.ai/api/v1/models/<id>/endpoints (public, no key): the dearest endpoint, because OpenRouter picks the provider per request. Includes regional, fast and priority endpoints, prompt-length tiers and audio input, at the undiscounted price where a discount is listed |
| Anthropic, OpenAI SDK compatibility (claude-sonnet-5-5, …) | 18 | Anthropic pricing: fast mode where offered, times the 1.1x inference_geo multiplier on Claude 4.6 and later |
| DeepSeek API (deepseek-flash, deepseek-v4-pro) | 4 | DeepSeek pricing, peak band, cache-miss input |
| Gemini API, OpenAI compatibility (gemini-3.6-flash, …) | 11 | Gemini pricing: Priority tier, and audio input where it bills above text |
| OpenAI and BlockRun | 7 | OpenAI published prices; BlockRun's free nvidia/nemotron-3.5-bolt ($0 for unpaid calls only) |
What the rule produces, for example:
- Gemini API. The compatibility layer honours
service_tier="priority"(1.8x) andinput_audiofrom the request body, sogemini-2.5-flashis $1.80/$4.50 (Priority, audio input) where the standard text price is $0.30/$2.50. - Anthropic layer. It documents neither fast mode nor
inference_geo, but it does forward some extra body fields (thinking), so both are priced in:claude-sonnet-5-5is $2.20/$11 (standard $2/$10, times 1.1), andclaude-fable-5-2is $17.60/$88 (fast mode $16/$80, times 1.1). - Prompt-length tiers.
claude-haiku-5-5uses its over-100k-prompt price.claude-sonnet-4-5-20250929uses $6/$22.50, the over-200k price that OpenRouter lists for Anthropic's own AWS endpoint (Anthropic's page lists $3/$15).claude-mythos-previewhas no published price and stays on the fallback. - OpenRouter.
anthropic/claude-sonnet-4.6is $3.30/$16.50 because regional Bedrock and Vertex bill 1.1x.google/gemini-3.7-flashis $2.70/$13.50, its priority endpoint without the listed 50% discount.anthropic/claude-fable-5.2is $16/$80 because a fast-mode endpoint is listed. - Announced increases.
gemini-3.6-flashandgemini-3.8-flashcarry the Priority price Google announced for 2027-01-01 ($2.70/$13.50), andgemini-4-carbon/gemini-4-argonthe one for 2026-12-01 ($7.20/$36). - Rows above the fallback. Four rows bill more than $60 output (up to $88)
and keep their price. So does an id that extends one of them with
-or:(anthropic/claude-fable-5.2:nitro,claude-fable-5-2-20261201), and the same applies to an operator override. A variant of a row below the fallback still gets $15/$60. - Legacy and retired ids.
deepseek-v4-flashanddeepseek-v4-flash-vision-expare legacy names that DeepSeek still accepts and bills at the Flash price.deepseek-chat,deepseek-reasonerand the retired OpenRouter idsdeepseek/deepseek-r1,anthropic/claude-3.5-sonnetandgoogle/gemini-2.0-flash-001are not listed by their providers and stay on the fallback. Configure a verified rate if your upstream still serves them.
The fallback is not a ceiling. An unlisted model that bills more than $60 per
million output tokens is under-counted until you add an override. Settlement
prices prompt_tokens + completion_tokens from the upstream's usage block, so
per-image, per-search and per-request fees are not counted, and neither are
reasoning tokens a provider leaves out of completion_tokens. Static rates are
estimates, not live prices; no catalog fetch runs in the request path.
