@orqo/foundationmodels-server

OpenAI-compatible HTTP server for Apple Foundation Models via @orqo/foundationmodels.

Install

pnpm add @orqo/foundationmodels-server

CLI usage

npx foundationmodels-server --port 3000

Options:

--port <n>                 Port to listen on (default 3000)
--host <host>              Host to bind (default 127.0.0.1)
--https                    Serve over TLS (requires --cert and --key)
--cert <path>              Path to the TLS certificate (PEM)
--key <path>               Path to the TLS private key (PEM)
--bearer-token <tok>       Bearer token (process-visible; prefer env/file)
--bearer-token-file <path> Read bearer token from file (trailing newline trimmed)
--require-auth             Fail to start if no bearer token is configured
--cors                     Enable permissive CORS headers (off by default; wildcard —
                           prefer the `cors: string[]` allowlist when embedding, see below)

Bearer token (TCK-0242 / FND-0224)

Putting secrets on argv leaks them via ps/top//proc/*/cmdline. Prefer env or file:

| Source | How | Notes | |---|---|---| | env (preferred) | FOUNDATIONMODELS_SERVER_BEARER_TOKEN | Not visible in process listings | | file | --bearer-token-file /run/secrets/token | File contents trimmed | | flag (back-compat) | --bearer-token <tok> | Emits a one-line stderr WARNING |

Precedence: --bearer-token > --bearer-token-file > FOUNDATIONMODELS_SERVER_BEARER_TOKEN.

# Preferred
export FOUNDATIONMODELS_SERVER_BEARER_TOKEN="$(cat /run/secrets/token)"
npx foundationmodels-server --port 3000 --require-auth

# Or file
npx foundationmodels-server --bearer-token-file /run/secrets/token --require-auth

Also: FOUNDATIONMODELS_SERVER_REQUIRE_AUTH=1 same as --require-auth.

Endpoints: POST /v1/chat/completions, GET /v1/models, GET /health, GET /metrics.

Input hardening (TCK-0248 / FND-0230)

POST /v1/chat/completions validates the body before it reaches the runtime:

• Shape/ranges — messages required; roles restricted; model must match a FoundationModelId; temperature ∈ [0, 2]; top_p ∈ (0, 1]; token caps positive integers; stop / stream typed.
• JSON bounds — after parse, nesting depth (default 32) and total object keys (default 10_000) are capped. Override via ServerOptions.maxJsonDepth / maxJsonKeys. Byte cap remains maxBodyBytes (default 10 MB → 413).

Metrics (TCK-0263 / FND-0245)

GET /metrics scrapes the in-process RED registry (fm.metrics()):

| Request | Response | |---|---| | GET /metrics | JSON FoundationModelsMetricsSnapshot | | GET /metrics?format=prometheus | Prometheus text exposition (text/plain) |

Unauthenticated (same posture as /health). Put a reverse proxy in front if scrapers must be gated.

HTTP example

curl http://127.0.0.1:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "apple.system",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'

# Streaming
curl http://127.0.0.1:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"apple.system","stream":true,"messages":[{"role":"user","content":"Hello!"}]}'

Programmatic usage

import { createFoundationModels } from "@orqo/foundationmodels";
import { startServer, createServer, createRequestHandler } from "@orqo/foundationmodels-server";

const fm = await createFoundationModels();

// Start and bind a server in one call.
const server = await startServer(fm, {
  port: 3000,
  host: "127.0.0.1",
  bearerToken: process.env.API_TOKEN,
  cors: true,
});
console.log("Listening on port 3000");

// Or embed the handler into an existing Node.js http.Server.
import { createServer as httpCreateServer } from "node:http";
const handler = createRequestHandler(fm, { bearerToken: process.env.API_TOKEN });

Auth and CORS (TCK-0218)

requireAuth is enforced at construction, on every entry point. Building a handler with requireAuth: true (or FOUNDATIONMODELS_SERVER_REQUIRE_AUTH=1) and no bearerToken throws synchronously from createRequestHandler, createServer and startServer alike.

Breaking change (FND-0179). Until this release the check lived only in startServer, so the embed path — createServer / createRequestHandler, the one this README recommends — happily served /v1/* unauthenticated with requireAuth set. If your code relied on that, it was serving an open endpoint while believing otherwise; the throw is the fix, not a regression.

cors accepts an allowlist. cors: true is unchanged (wildcard Access-Control-Allow-Origin: * on every response) and now logs a one-time warning, because a wildcard on an authenticated endpoint invites the browser to do exactly what the token is there to prevent. Prefer:

const handler = createRequestHandler(fm, {
  bearerToken: process.env.API_TOKEN,
  cors: ["https://app.example.com"],  // exact-match reflection, sets Vary: Origin
});
const httpServer = httpCreateServer((req, res) => void handler(req, res));
httpServer.listen(3001);

Author

Cristiano Aredes — sole author and maintainer.

• GitHub: @cristianoaredes
• X: @CristianoAredes
• LinkedIn: Cristiano Aredes

Part of FoundationModels JS.

License

AGPL-3.0-only — see LICENSE. Copyright © 2026 Cristiano Aredes.

Network clause (AGPL §13): modified versions offered as a network service must provide Corresponding Source to users of that service.