@overskyai/inference-broker

A loopback credential broker. It keeps a managed-inference credential out of the process that uses it.

Why this exists

A managed OverSky session runs a coding harness — a model-driven process with tool execution — and that harness needs to reach an inference endpoint. The obvious implementation hands it a bearer token through the environment. That is the one thing this package exists to prevent, because the harness's environment is also available to its descendants, to crash dumps, and to the default wrapper's serialized request.

Instead the broker listens on a Unix-domain socket in the session's own temp directory, and the harness is given a base URL on that socket plus a loopback-only secret that is worthless anywhere else. The broker holds the real session capability, attaches it on the way out, and enforces the request envelope the server authorized before anything leaves the machine.

The property this buys is stated as invariant I1 in docs/architecture/managed-inference-execution-shapes-2026-08-12.md:

No cloud-reusable credential enters the harness process. Not its environment, not its address space, not a file it can read.

That invariant is deliberately implementation-free, which is why this is a package rather than a directory in one service: it is satisfied by a broker hosted in the daemon (cloud-dispatched runs) or in the CLI (a human running sky code), and both hosts run the same code. Before the extraction, only the daemon could host one, which made a background service a hard prerequisite for an interactive command.

What it does

• Serves a loopback relay over a Unix socket, in the provider dialect the admitted model needs (OpenAI-compatible, native Anthropic Messages, or Bedrock).
• Enforces the request envelope locally before egress — request count, in-flight concurrency, body size — so a violation fails fast instead of at the metering gateway. The server re-checks everything; this is defence in depth, not the control.
• Runs the lease renewal heartbeat, extending the session's authority while the human is present and classifying every refusal into a typed RenewalStopReason (stopped, denied, unavailable, ceiling, expired) so the caller can say which ending this was.
• Refreshes the session capability it forwards with, swapping the one variable every forwarded request and every lease renewal reads.
• Reports status on a reserved local path so a supervised session's spend and remaining lease can be shown to the human in real time — invariant I5.

What it is not

It is not a proxy you point at arbitrary hosts. Every request is checked against the daemon-issued broker origin and refused otherwise, and the socket path is supplied for exactly one spawned session — there is no socket discovery, no proxy environment, and no remote fallback.

Consumers

daemon (cloud-dispatched managed sessions) and @overskyai/cli (sky code). Both depend on this package; neither reimplements it.

Testing

npm test        # node --import tsx --test 'src/**/*.test.ts'

The suite covers the loopback server, dialect routing, request-envelope enforcement, the renewal loop and its refusal classification, and the status channel. It is the only coverage for any of those behaviours anywhere in the repository, so it runs in CI as its own job rather than under the monorepo's jest projects — the runner is Node's built-in test runner, not jest.