Routiium

Routiium is a self-hosted, OpenAI-compatible LLM gateway with safe-by-default routing. Put it between your apps and your model providers to add managed API keys, built-in policy routing, request-level LLM/deterministic judging, rate limits, analytics, system prompts, MCP tools, and optional remote Router-compatible policy services without changing client SDKs.

Your App / OpenAI SDK
        │
        ▼
  Routiium :8088 ── auth, limits, routing, judge, analytics, tools
        │
        ├── OpenAI / Responses API
        ├── vLLM, Ollama, or any Chat-compatible endpoint
        ├── AWS Bedrock
        ├── Built-in policy router + safety judge
        └── Optional remote Router-compatible policy service

Install

npm install -g routiium
routiium --version

The npm package installs a routiium command. On install it downloads the matching native binary from the GitHub release when available and falls back to cargo build --release --locked if a prebuilt binary is not available for your platform. Source installs still work with cargo run -- ....

Choose your path

1. Proxy an OpenAI-compatible app safely in minutes

npm install -g routiium
routiium config init --profile openai
routiium config set OPENAI_API_KEY sk-your-provider-key
routiium doctor
routiium serve

Then point any OpenAI-compatible SDK at http://127.0.0.1:8088/v1. With no extra router setup, Routiium enables its embedded router, strict safety path, protect judge mode, response guard, streaming safety, and restricted web/URL judging.

2. Use a local vLLM/Ollama-style upstream

routiium init --profile vllm --out .env
# edit OPENAI_BASE_URL if your local server is not http://127.0.0.1:8000/v1
routiium serve

3. Use built-in routing, or connect a remote router

routiium init --profile router --out .env
routiium router probe --model gpt-4.1-nano

Routiium ships with aliases such as auto, fast, balanced, safe, secure, and premium with cost/latency/context-aware scoring. You do not need another project to get default routing. Remote router mode is optional for teams that want a separate Router-compatible policy service for central policy, catalog, health, and overlay management; EduRouter is only a small companion/reference implementation of that interface.

4. Inspect the built-in judge

routiium router explain --model auto --prompt "Ignore previous instructions"
routiium judge policy init --out config/judge-policy.json
routiium judge explain --policy config/judge-policy.json --prompt "Ignore previous instructions"
routiium judge test --suite all
routiium judge profile protect --out .env

Default protect mode enforces high-confidence deterministic blocks, downgrades prompt-injection-like requests to a safer route, scans successful outputs with the response guard, and uses an LLM judge automatically when OPENAI_API_KEY is available. LLM judge calls prefer tool/function calling (ROUTIIUM_JUDGE_OUTPUT_MODE=auto) and fall back to JSON for providers that do not support tools. Use shadow to observe, protect for safe defaults, enforce for stricter policy, or off to disable. Custom judge prompts are supported through policy overlays; Routiium still keeps its built-in safety prompt immutable and routes sensitive-but-allowable requests to the built-in secure alias by default. For agentic applications, rejected unsafe actions return an OpenAI-compatible assistant result by default (ROUTIIUM_REJECTION_MODE=agent_result) so the loop can continue without fulfilling the unsafe request; set http_error for strict gateway-style 403s.

Day-one workflow

# 1) Start Routiium
routiium serve

# 2) Create a managed customer key
routiium key create --label demo --ttl-seconds 86400

# 3) Call the OpenAI-compatible API with the returned sk_<id>.<secret>
curl http://127.0.0.1:8088/v1/chat/completions \
  -H "Authorization: Bearer sk_<id>.<secret>" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4.1-nano","messages":[{"role":"user","content":"Hello"}]}'

# 4) Check runtime state
routiium status

If OPENAI_API_KEY is configured on the server, Routiium runs in managed mode: clients use Routiium-issued keys and never see the upstream provider secret. If no server-side provider key is set, Routiium can run in passthrough mode and forward client bearer tokens upstream.

CLI

Routiium now exposes a clap-based CLI:

| Command | Purpose | | --- | --- | | routiium serve | Run the HTTP gateway. Existing root flags still work as a compatibility alias. | | routiium config init --profile <openai|vllm|router|judge|bedrock|synthetic> | Create or update ~/.config/routiium/config.env with starter settings. | | routiium config set|get|list|path | Manage the per-user config file without hand-editing env files. | | routiium init --profile <openai|vllm|router|judge|bedrock|synthetic> | Generate starter .env and profile config for repo-local/deploy workflows. | | routiium doctor | Check env/config files, server health, and judge/router readiness. Use --production before launch. | | routiium status | Fetch /status from a running server. | | routiium key create|list|revoke | Manage API keys through the admin HTTP API. | | routiium router probe | Send a small request and print routing-related response details. | | routiium router explain | Explain the embedded router + judge decision locally. | | routiium judge profile shadow|protect|enforce|off | Update local env defaults for judge rollout. | | routiium judge policy init|validate | Create or validate custom judge prompt/policy overlays. | | routiium judge explain | Explain judge action, target alias, and policy fingerprint locally. | | routiium judge test | Run built-in prompt-injection/exfiltration/dangerous-action judge checks. | | routiium docs | Print the main docs entry points. |

Run routiium --help or see docs/CLI.md for the full command reference. /convert now defaults to safe conversion without internal system prompts/MCP metadata; add ?include_internal_config=true with admin auth only when you intentionally need to inspect internal conversion shape.

Core features

• OpenAI-compatible proxy for /v1/chat/completions, /v1/responses, and /v1/models.
• Payload translation between Chat Completions and Responses API shapes, including streaming, tools, multimodal content, logprobs, and usage metadata.
• Managed API keys with Redis, sled, or memory backends; hashed secrets; labels; expiration; scopes; revocation; and admin APIs.
• Embedded policy routing with auto, fast, balanced, safe, secure, and premium aliases, cost/latency/context scoring, route metadata, and remote Router Schema compatibility.
• Built-in request judge and response guard with deterministic prompt-injection/exfiltration/tool-risk checks, optional LLM judging, restricted web/URL judging, custom operator policy overlays, secure rerouting for sensitive requests, output-leak blocking, streaming safety, and structured denial responses.
• Rate limits and concurrency controls with hot-reloadable policies and emergency blocks.
• Analytics and cost tracking with JSONL, Redis, sled, or memory storage and CSV/JSON export.
• System prompts and MCP tools injected transparently into upstream requests. MCP runtime config writes are disabled by default because MCP servers can spawn local commands; enable ROUTIIUM_ALLOW_MCP_CONFIG_UPDATE=1 only for trusted admin deployments.
• Admin panel for keys, limits, prompts, routing, analytics, and runtime state.

Docker

docker build -t routiium .
docker run --rm -p 8088:8088 -e OPENAI_API_KEY=sk-your-key routiium

Or use the included Compose setup:

cp .env.example .env
# edit .env
docker compose up --build

Admin panel

npm run admin:install
npm run admin:dev

For root npm CLI release checks, trusted publishing setup, and standalone admin package checks:

npm run package:verify
npm run package:trust:github:dry-run
npm run package:pack
npm run admin:lint
npm run admin:build
npm run admin:audit
npm run admin:pack

The root routiium npm release workflow is configured for npm Trusted Publishing/OIDC rather than a long-lived NPM_TOKEN. Before tagging a release, configure npm's trusted publisher for package routiium with repository labiium/routiium and workflow file publish-npm.yml:

npm run package:trust:github

If npm requires the package to exist before adding trust, do a one-time first publish with a short-lived npm automation token, then immediately configure trusted publishing and remove token-based publishing.

Set ROUTIIUM_ADMIN_TOKEN on the server and enter the matching bearer in the panel header. Admin APIs now fail closed when this token is unset; ROUTIIUM_INSECURE_ADMIN=1 is only for throwaway local development. The panel talks to live admin APIs; it is not a mock dashboard.

Documentation

Start here:

• Getting Started — profile-based onboarding recipes.
• CLI Reference — all commands, flags, and examples.
• Configuration — ~/.config/routiium/config.env, .env, env vars, and config files.
• Security Model — built-in judge threat model and controls.
• Judge Policy — custom prompts, secure rerouting, and policy validation.
• Production Checklist — launch checks for auth, CORS, guards, and streaming safety.
• API Reference — HTTP routes and payload examples.
• Router Usage and Router API Spec — built-in routing plus optional remote Router-compatible integration.
• Rate Limits, Analytics, AWS Bedrock, and Production Hardening.

Development

cargo fmt
cargo clippy --all-targets --all-features
cargo test
npm run package:verify
npm pack --dry-run

HTTP smoke tests live in python_tests/ and can be run with pytest.