service-llm

Use your ChatGPT Codex subscription through OpenAI-compatible HTTP APIs or MCP.

service-llm supports Chat Completions, Responses, SSE streaming, function calls, and an MCP ask tool. It returns model tool calls but never executes them.

Requirements

• Node.js 22.19 or newer
• A ChatGPT account with Codex access

Install

npm install --global @leoustc/service-llm

Show the installed CLI version:

service-llm -v

Use service-llm --help to show all available commands and server options.

Log in

service-llm login

For a headless or remote machine:

service-llm login --device-code

Credentials are stored securely at ~/.service-llm/auth.json. During login, the service fetches the available Codex API models and records them at ~/.service-llm/models.json.

If you start the service before logging in, the first model request returns a device code and verification URL. Complete the browser login, then retry the request.

Choose a model

List supported models and show the current default:

service-llm model

The default is marked with *. Set a different default with:

service-llm model set gpt-5.4-mini

An explicit valid model in an API request overrides this default for that request.

Start the server

service-llm serve

The server listens on 127.0.0.1:7060 by default.

At startup it prints the version, listening URL, authentication state, default model, supported-model count, endpoint URLs, and debug status. API key values and OAuth credentials are never displayed.

Choose a different address or port:

service-llm serve --host 0.0.0.0 --port 8080

Protect the API with one or more client keys:

service-llm serve --api-key secret-one,secret-two

You can also load the listening address, port, and client keys from JSON:

{
  "host": "0.0.0.0",
  "port": 8080,
  "api_key": "abc12345"
}

Multiple keys:

{
  "port": 8080,
  "api_keys": ["key-one", "key-two"]
}

chmod 600 service-llm.json
service-llm serve -f service-llm.json

Use api_key for one key, or api_keys for an array or comma-separated string. Explicit --host, --port, and --api-key options override the config file. The config file overrides SERVICE_LLM_API_KEYS.

Clients may send a key through either header:

Authorization: Bearer secret-one

X-API-Key: secret-one

Do not expose the server on a public interface without authentication and appropriate network controls.

Chat Completions

curl http://127.0.0.1:7060/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5.4",
    "messages": [
      {"role": "user", "content": "Write a short story."}
    ]
  }'

Add "stream": true to receive OpenAI-compatible SSE chunks.

The compatibility layer supports text and image message content, developer and system messages, function tools, named or automatic tool choice, reasoning effort, verbosity, response formats, parallel tool-call control, and streamed usage. Codex does not provide every optional Chat Completions feature; unsupported sampling, output-token limits, and storage options are not emulated.

Responses API

curl http://127.0.0.1:7060/v1/responses \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5.4",
    "input": "Write a short story."
  }'

Add "stream": true to receive typed Responses SSE events.

Responses requests also honor top-level instructions, reasoning.effort, and text.verbosity, along with text formats, truncation, parallel tool-call control, function and custom tool calls, tool-output continuations, image inputs, and encrypted reasoning items. The compatibility endpoint accepts and reports max_output_tokens, but the Codex subscription transport does not enforce it. When omitted, the configured low reasoning effort and low verbosity defaults are used. The service uses stateless upstream requests (store: false); stored-response retrieval, deletion, background responses, and hosted OpenAI tools are not emulated.

GPT-5.6 compatibility

GPT-5.6 models may produce several assistant message items interleaved with reasoning or tool-call items in one response. This is valid Responses API output, not malformed model output. service-llm preserves those item boundaries and their original order, emits a complete SSE lifecycle for each item, and separates the message segments when adapting them to Chat Completions.

Codex App Server may also supply its local tools through an additional_tools input item. The service forwards those tool definitions to the model while continuing to leave execution to the Codex client.

Function calls

Supply OpenAI-compatible function or custom tools and tool_choice. The model may return function_call or custom_tool_call output items, but service-llm will not execute them. Return the model output items together with the matching function_call_output or custom_tool_call_output in the next request's input; custom tools use plain-text input instead of JSON arguments.

curl http://127.0.0.1:7060/v1/responses \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5.4",
    "input": "Write a short story and submit it with save_story.",
    "tools": [{
      "type": "function",
      "name": "save_story",
      "description": "Save a generated story",
      "parameters": {
        "type": "object",
        "properties": {
          "title": {"type": "string"},
          "story": {"type": "string"}
        },
        "required": ["title", "story"]
      }
    }],
    "tool_choice": "required"
  }'

MCP

Connect an MCP client to:

http://127.0.0.1:7060/mcp

The server provides two tools:

• ask: accepts prompt and optional model, tools, and tool_choice arguments.
• list_models: lists available models and reports the currently configured default model and reasoning level.

The endpoint implements stateless MCP Streamable HTTP with JSON-RPC 2.0, protocol negotiation through MCP revisions 2025-03-26, 2025-06-18, and 2025-11-25, notification handling, legacy batch input for 2025-03-26 clients, standard tool errors, and browser Origin validation. It returns JSON responses directly and does not open a server-initiated SSE channel.

Example JSON-RPC request:

curl http://127.0.0.1:7060/mcp \
  -H 'Content-Type: application/json' \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "ask",
      "arguments": {"prompt": "Write a short story."}
    }
  }'

Model selection

Specify a model with the request's model field. If it is omitted or unavailable, the service uses your configured default. On first login, the lowest-cost available model is selected with low reasoning effort. Responses always report the model actually used.

List the models available to both Chat Completions and Responses clients:

curl http://127.0.0.1:7060/v1/models

When client authentication is enabled, include the same Bearer or X-API-Key header used for generation requests. The response includes default_model and default_reasoning_effort in addition to the OpenAI-compatible model list.

Debug logging

Enable request, response, and SSE logs with:

DEBUG=service-llm service-llm serve

Authentication headers are redacted. Prompts and model output are not redacted, so avoid debug logging for sensitive requests.

Unexpected upstream or internal failures return HTTP 503 with {"status":"busy"}. Raw exception and provider details are written only to the server log and are never included in the client response. Streaming endpoints emit the same generic busy status before closing.

Health check

curl http://127.0.0.1:7060/health

Expected response:

{"status":"ok"}

License

Apache License 2.0. See LICENSE.