@x12i/ai-dispatcher

One request shape for OpenRouter, AWS Bedrock, the OpenAI Responses API, and Cloudflare AI. You send the same request shape. The package picks the provider, and @x12i/ai-profiles@^5.1.0 turns reasoningEffort into that provider's wire fields. Callers do not map effort levels onto provider fields or parse provider reasoning channels.

Implemented providers: openrouter, bedrock, openai, cloudflare. Any other provider throws PROVIDER_NOT_IMPLEMENTED.

The package is @x12i/ai-dispatcher 2.3.1. It runs on Node 20 or newer and publishes ESM and CommonJS from the same entry.

Connected MCP servers can be passed to createAiDispatcher. run() and compile() expose those tools to the model. executeStreamingChat does not. Details are in MCP tools.

Install

npm install @x12i/ai-dispatcher

The package depends on @x12i/ai-profiles@^5.1.0, @x12i/provider-metadata@^1.2.0, @x12i/openrouter-runtime@^2.2.0, and @x12i/bedrock-runtime@^2.2.0. registerHostIntents is re-exported from this package so a host registers intents on the same @x12i/ai-profiles copy the dispatcher resolves with.

One request

import { createAiDispatcher } from "@x12i/ai-dispatcher";

const dispatcher = createAiDispatcher({
  openrouter: { apiKey: process.env.OPENROUTER_API_KEY },
  bedrock: { region: process.env.AWS_REGION },
  openai: { apiKey: process.env.OPENAI_API_KEY },
  cloudflare: {
    apiToken: process.env.CLOUDFLARE_API_TOKEN,
    accountId: process.env.CLOUDFLARE_ACCOUNT_ID
  }
});

const response = await dispatcher.run({
  provider: "openai",
  model: "gpt-5.4",
  prompt: "Explain the tradeoff in one paragraph.",
  reasoningEffort: "low"
});

console.log(response.text);
console.log(response.reasoningText);
console.log(response.reasoningResolution);

Provider selection is request.provider, then options.provider, then openrouter. Model selection is request.model, then that provider's defaultModel.

openai sends POST {baseUrl}/responses. The default base URL is https://api.openai.com/v1, and the catalog reasoning object is placed on that Responses body.

Use model ids that belong to the selected catalog target. An OpenRouter id is not a Bedrock id and is not an OpenAI id.

| Target | Example model id | reasoningEffort: "low" on the wire | | --- | --- | --- | | openrouter | openai/gpt-5.2 | reasoning.effort: "low" on the chat-completions body | | bedrock | anthropic.claude-fable-5-1 | additionalModelRequestFields.thinking.type: "adaptive" and output_config.effort: "low" | | openai | gpt-5.4 | reasoning.effort: "low" on the Responses body | | cloudflare | openai/gpt-5.4 | reasoning.effort: "low" on the Cloudflare Responses body |

Representative catalog results for @x12i/[email protected]:

• OpenRouter ~google/gemini-flash-latest + max is degraded and sends reasoning.effort: "high".
• OpenRouter aion-labs/aion-2.0 is ignored with an empty reasoning body. The call still runs.
• Bedrock anthropic.claude-fable-5-1 + max is degraded and sends output_config.effort: "high".
• OpenAI gpt-5.4 + max is applied and sends reasoning.effort: "max".
• OpenAI gpt-4o is ignored. The Responses call still runs.

The type AiProviderId also lists anthropic, google, azure, groq, mistral, cohere, together, fireworks, xai, deepseek, and custom. Passing any of those throws PROVIDER_NOT_IMPLEMENTED.

Configuration

createAiDispatcher({
  provider: "openrouter",
  logger: {
    info: (event, data) => console.log(event, data)
  },
  openrouter: {
    apiKey: process.env.OPENROUTER_API_KEY,
    defaultModel: "openai/gpt-5.2"
  },
  bedrock: {
    region: process.env.AWS_REGION ?? "us-east-1",
    defaultModel: "anthropic.claude-fable-5-1"
  },
  openai: {
    apiKey: process.env.OPENAI_API_KEY, // or OPENAI_API_KEY
    baseUrl: "https://api.openai.com/v1",
    defaultModel: "gpt-5.4",
    organization: process.env.OPENAI_ORG_ID,
    project: process.env.OPENAI_PROJECT_ID,
    timeoutMs: 60_000,
    maxAttempts: 2
  },
  cloudflare: {
    apiToken: process.env.CLOUDFLARE_API_TOKEN,
    accountId: process.env.CLOUDFLARE_ACCOUNT_ID,
    gatewayId: "default",
    defaultModel: "@cf/meta/llama-3.1-8b-instruct",
    endpoint: "auto",
    timeoutMs: 120_000,
    maxAttempts: 2
  }
});

dispatcher.provider is the default provider id. Adapters are created on first use and reused. The default provider's adapter is created immediately.

The logger callback is { debug, info, warn, error }. A provider-specific logger on openrouter, bedrock, openai, or cloudflare wins over the dispatcher logger for that adapter. You can inject fetch on OpenAI, OpenRouter, and Cloudflare, and a client on Bedrock.

Missing OpenAI credentials throw OPENAI_API_KEY_MISSING. Missing Bedrock region throws the runtime's REGION_REQUIRED. Missing OpenRouter key throws OPENROUTER_API_KEY_MISSING. Missing Cloudflare token throws CLOUDFLARE_API_TOKEN_MISSING. Missing Cloudflare account id throws CLOUDFLARE_ACCOUNT_ID_MISSING.

Credentials and transport

| Provider | Required | Default transport | | --- | --- | --- | | OpenRouter | openrouter.apiKey | OpenRouter chat completions | | Bedrock | bedrock.region | Converse / ConverseStream. Credentials are the AWS default chain, or bedrock.credentials | | OpenAI | openai.apiKey or OPENAI_API_KEY | POST {baseUrl}/responses. Default base URL is https://api.openai.com/v1. Default timeout is 120 seconds. Default attempts is 2 | | Cloudflare | cloudflare.apiToken or CLOUDFLARE_API_TOKEN, and cloudflare.accountId or CLOUDFLARE_ACCOUNT_ID | POST https://api.cloudflare.com/client/v4/accounts/{accountId}/ai/.... Default timeout is 120 seconds. Default attempts is 2 |

Optional OpenAI headers are OpenAI-Organization and OpenAI-Project.

Cloudflare uses one account token for Workers AI (@cf/...) and third-party models (openai/..., anthropic/..., and the rest). auto sends anthropic/ models to /ai/v1/messages, openai/ and bare gpt or o1/o3/o4 ids to /ai/v1/responses, and every other id, including @cf/, to /ai/v1/chat/completions. Set cloudflare.endpoint to chat, responses, messages, or run to force a path. That field wins over apiMode. apiMode: "chat" and "responses" still select those two paths when the endpoint is auto. Workers AI always sends cf-aig-gateway-id, using default when gatewayId is omitted. Third-party models omit that header unless gatewayId is set. /ai/run wraps the chat payload as { model, input }. run.webhookUrl requires run.background: true. executeStreamingChat rejects run because that endpoint is synchronous or webhook, not SSE.

Cloudflare retries 429, 408, and 5xx with backoff capped at 5 seconds. After the last retryable failure, run() returns a failed response and streaming yields stream.error and then throws. 401 and 403 become PROVIDER_AUTH_FAILED. 404 becomes PROVIDER_MODEL_NOT_FOUND. A last 429 becomes PROVIDER_RATE_LIMITED. Other HTTP failures, including a Cloudflare { success: false, errors: [...] } body, become PROVIDER_REQUEST_FAILED. Timeouts and network failures are PROVIDER_GATEWAY_UNREACHABLE. Config mistakes (CLOUDFLARE_API_TOKEN_MISSING, CLOUDFLARE_ACCOUNT_ID_MISSING, MODEL_REQUIRED, INPUT_REQUIRED, UNSUPPORTED_REQUEST_FIELD) throw from run() instead of becoming a failed response.

OpenAI retries 429, 408, and 5xx with backoff capped at 5 seconds. After the last retryable failure, run() returns a failed response and streaming yields stream.error and then throws. 401 and 403 become PROVIDER_AUTH_FAILED. 404 becomes PROVIDER_MODEL_NOT_FOUND. Other HTTP failures become PROVIDER_REQUEST_FAILED. Timeouts and network failures are PROVIDER_GATEWAY_UNREACHABLE. Config mistakes (OPENAI_API_KEY_MISSING, MODEL_REQUIRED, INPUT_REQUIRED, UNSUPPORTED_REQUEST_FIELD) throw from run() instead of becoming a failed response.

The request

AiDispatcherRequest is the OpenRouter runtime request plus provider, orgId, stepId, skillId, reasoningEffort, bedrockControls, cache, cloudflare, and mcp. agentId is already on that request.

Text can arrive as prompt, messages, input, system, or instructions. A message role is system, user, assistant, or tool. Content is a string or parts: text, image_url, or file.

| Field | Meaning | | --- | --- | | id | Caller request id. Logged when set. OpenAI uses it as a fallback response id. | | agentId | Optional caller agent id. Stored in the one metadata record. On OpenRouter it is also the activity-log App name. | | orgId | Optional caller organization id. Stored in the one metadata record as orgId. | | stepId | Optional caller step id. Stored in the one metadata record as stepId. | | skillId | Optional caller skill id. Stored in the one metadata record as skillId. | | temperature, maxTokens | Sampling and output cap. OpenAI sends maxTokens as max_output_tokens. | | reasoning | Explicit wire object for OpenRouter, OpenAI, and Cloudflare. | | functionTools | Local function tools: name, description, parameters, optional executor. | | toolChoice | "auto", "none", "required", { type: "function", functionName }, or an OpenRouter { type: "server_tool", serverTool }. | | execution | maxToolIterations, maxServerToolCalls, maxFunctionToolCalls, timeoutMs. Bedrock keeps the iteration, function-call, and timeout limits. | | metadata | The one metadata record. String, number, and boolean values, including xtivix.execution, recordId, and objectType. | | serverTools | OpenRouter server tools only. Bedrock, direct OpenAI, and Cloudflare reject an enabled policy. | | apiMode | "chat", "responses", or "auto". Direct OpenAI rejects "chat". Bedrock rejects any apiMode. On Cloudflare, "chat" and "responses" select those endpoints when cloudflare.endpoint is omitted or auto. | | responseFormat | OpenRouter forwards it. Direct OpenAI and Cloudflare Responses place it on text. Cloudflare chat sends response_format. Cloudflare messages and run reject it. Bedrock rejects it. | | rawOpenRouterOverrides | Merged onto an OpenRouter body. A metadata object inside it is folded into metadata on every provider. Any other override key is rejected on Bedrock, direct OpenAI, and Cloudflare. | | cache | Optional prompt-cache intent: mode (auto, explicit, or disabled), key, and ttl (5m, 30m, or 1h). Omit it and the compiled body is unchanged. explicit places one breakpoint at the end of system or instructions when that model supports it. On Cloudflare it also sets AI Gateway cache headers. | | cloudflare | Per-call overrides: endpoint, gatewayId, run, and cf-aig-* controls. Stripped before compilation. | | mcp | false or [] omits MCP tools registered at initialization. A list of exposed names attaches only those tools. Omit the field to attach every registered MCP tool. |

Set metadata for the pairs you want back from a log. agentId, orgId, stepId, and skillId are fields on the same record. A role object expands to role, roleLayer, roleDigest, and profileChoice on that record. A dedicated field wins over the same key inside metadata. Omit an id you do not have. The dispatcher does not invent one.

import { decodeProviderMetadata } from "@x12i/ai-dispatcher";

await dispatcher.run({
  model: "openai/gpt-5.2",
  prompt: "Explain the tradeoff in one paragraph.",
  agentId: "agent-42",
  orgId: "org-7",
  stepId: "step-3",
  skillId: "skill-9",
  metadata: {
    "xtivix.execution": executionJson,
    recordId: "rec-1",
    objectType: "note"
  }
});

const record = decodeProviderMetadata(logMetadata);

metadata is the one record. When rawOpenRouterOverrides only repeats metadata, the dispatcher folds it in. metadata wins when the same key is in both places. String, number, and boolean values are stored, including a JSON string. Objects and arrays stay on the normalized response and the tool context.

Every provider writes that record as m.0, m.1, and so on: one base64 string, 256 characters per piece, at most 16 pieces. The base64 is the UTF-8 JSON object of the string map, with keys sorted. decodeProviderMetadata reads those pieces from OpenRouter body metadata, OpenAI Responses metadata, Bedrock requestMetadata, or Cloudflare body metadata. Pass the Cloudflare cf-aig-metadata header as that same object or as the raw JSON string. The pieces are the same on every provider. The dispatcher trims on every call. Pass metadataPriority (highest priority first) to choose what stays. The default keeps agentId, orgId, stepId, skillId, role, roleLayer, roleDigest, profileChoice, recordId, and objectType. A trimmed call adds a METADATA_RECORD_TRIMMED warning.

Repeating vendor fields normalize to one key. Prefer metadata.api_key_name over metadata.openrouter.api_key_name. normalizeMetadata and decodeProviderMetadata promote known aliases and keep the vendor key. Grow METADATA_NORMALIZATION_MAP as more shared fields appear.

OpenRouter also puts agentId in X-Title and X-OpenRouter-Title, ahead of appAttribution.appName and defaultHeaders. That header is the activity-log App name. appAttribution.siteUrl is still HTTP-Referer when set. The App header is not a second metadata record.

OpenAI compilation order: instructions wins, then system, then system messages joined with newlines. input is sent as written. Otherwise non-system messages are converted, and a string prompt is appended as a user item. Tool messages become function_call_output. Image parts become input_image. A file part with url becomes input_file file_url. A file part with data becomes input_file file_data. fileName becomes filename. A file part with neither url nor data throws UNSUPPORTED_REQUEST_FIELD.

Cloudflare Responses uses that same compilation. Chat and run use OpenAI chat messages. Messages uses Anthropic content blocks, with system kept as its own field. Chat, messages, and run reject a request that sets both messages and input. toolChoice: "none" omits tools. A file part with neither url nor data throws UNSUPPORTED_REQUEST_FIELD.

Bedrock receives id, model, messages, system, prompt, temperature, maxTokens, functionTools, a Bedrock-compatible toolChoice, the execution limits above, agentId, and metadata (sent as requestMetadata). instructions is used as system when system is empty. A string input is used as prompt when prompt and messages are empty. input items with a role and content become messages when messages is empty. Sending both messages and input items throws UNSUPPORTED_REQUEST_FIELD. An enabled server tool, responseFormat, a rawOpenRouterOverrides entry other than metadata, apiMode, or a server_tool tool choice also throws. toolChoice: "none" omits tools. A data:image/(png|jpeg|jpg|gif|webp);base64,... part is sent as Converse image bytes (jpg is sent as jpeg). Any other image URL, and any other non-text part, throws UNSUPPORTED_CONTENT.

Entrypoints

| Method | What it does | | --- | --- | | dispatcher.run(request) | Non-streaming call. Returns answer text, optional reasoning text, usage, tool results, and diagnostics. | | dispatcher.executeStreamingChat(request) | The only streaming entrypoint. Yields typed events. | | dispatcher.compile(request) | Same preparation as a live call, without sending. | | dispatcher.refreshMcp() | Reloads MCP servers that implement listTools. A failed reload keeps the previous list. |

compile() returns a provider payload plus requestedModel, effectiveModel, and reasoningResolution when reasoning was involved:

• OpenRouter: { provider, url, headers, body, warnings }
• Bedrock: { provider, modelId, input }
• OpenAI: { provider, url, headers, body, warnings }
• Cloudflare: { provider, endpoint, url, headers, body, warnings }

Authorization, x-api-key, and api-key are replaced with [redacted] in the compiled result. Execution sends the real headers. A streaming call adds only the transport stream flag (stream: true, or Bedrock ConverseStream).

MCP tools

Pass connected MCP servers to createAiDispatcher. The dispatcher does not spawn processes or open transports. It lists the tools you already fetched and calls callTool when the model uses one.

const dispatcher = createAiDispatcher({
  openai: { apiKey: process.env.OPENAI_API_KEY },
  mcp: {
    servers: [{
      id: "fs",
      tools: [{
        name: "read_file",
        description: "Read a file",
        inputSchema: { type: "object", properties: { path: { type: "string" } } }
      }],
      callTool: (name, args) => session.callTool({ name, arguments: args }),
      listTools: async () => (await session.listTools()).tools
    }]
  }
});

await dispatcher.run({ model: "gpt-5.4", prompt: "Read notes.txt" });

Each tool is exposed as ${serverId}__${toolName}. callTool still receives the raw MCP name. Server ids and tool names must match ^[A-Za-z0-9_-]+$, and the exposed name must be at most 64 characters. A missing description is sent as an empty string. A missing inputSchema is sent as { type: "object", properties: {} }.

run() and compile() attach every registered tool. executeStreamingChat does not attach them and does not run them. Set mcp: false or mcp: [] on a request to attach none. Set mcp: ["fs__read_file"] to attach that tool only. An unknown name in that list throws MCP_TOOL_NOT_FOUND.

A functionTools entry with the same exposed name replaces the MCP tool for that call. If that entry has no executor, the MCP callTool does not run. An executor already stored on openrouter.tools, bedrock.tools, openai.tools, or cloudflare.tools under the exposed name runs instead of callTool.

Text blocks in an MCP result are joined and returned to the model. isError: true fails that tool call. Any other result is JSON. Bedrock toolChoice: "none" still omits tools. OpenRouter advisor, subagent, and fusion cannot call these sessions.

await dispatcher.refreshMcp() reloads servers that implement listTools and leaves the others unchanged. Overlapping reloads run one after another. A failed reload, including a list that breaks the name rules, throws MCP_LIST_FAILED and keeps the previous catalog. A call already in progress keeps the tools it prepared.

Reasoning effort

reasoningEffort is minimal | low | medium | high | max. extra-high is rejected with INVALID_EFFORT. It is not stored as max.

| Request | Catalog | | --- | --- | | No reasoningEffort and no explicit control | Not called. The model id is forwarded as written, including ids the catalog does not know. | | reasoningEffort only | resolveReasoning({ target, model, effort }) once for the original model. A catalog replacement model is the effective model and is not resolved again. | | Explicit control only | Kept. The catalog is not called. | | Both | The explicit control wins. The catalog body, model switch, and prompt prefix are not applied. reasoningResolution.bypassed is true, and outcome is omitted. |

Explicit controls are transport fields:

• OpenRouter, OpenAI, and Cloudflare: request.reasoning.
• Bedrock: request.bedrockControls.additionalModelRequestFields.

An OpenRouter reasoning object on a Bedrock call is not a Bedrock control. With reasoningEffort, the catalog still runs and that object is removed. Without reasoningEffort, the call fails with UNSUPPORTED_REQUEST_FIELD.

Outcomes:

• applied — the catalog wrote the requested effort.
• degraded — the catalog wrote a supported nearby effort. The call still runs.
• ignored — the model has no reasoning control. The call still runs with an empty catalog body.

Unknown models and missing mappings throw. The error keeps the catalog code and details, including UNKNOWN_MODEL and REASONING_MAPPING_MISSING. reasoningEffort without any model id throws REASONING_MODEL_REQUIRED.

Cloudflare has no catalog target of its own. reasoningResolution.target stays "cloudflare". The catalog is queried as openai after stripping an openai/ prefix (and bare gpt or o1/o3/o4 ids are prefixed back to openai/ when the catalog rewrites the model). Every other id, including anthropic/ and @cf/, is queried as openrouter. UNKNOWN_MODEL and REASONING_MAPPING_MISSING become outcome ignored with an empty body, so a Workers AI model still runs. The other providers still throw those codes.

The dispatcher does not rank efforts, invent token budgets, or rewrite maxTokens. Profile notes about output-token headroom stay on the catalog profile. The directive body already contains companion fields the wire needs, and those fields are merged onto the compiled body.

Answers and reasoning text

response.text is answer text. response.reasoningText is set only when the provider returned reasoning text. It is omitted when there is none.

response.usage keeps the runtime usage fields: inputTokens, outputTokens, totalTokens, costUsd, and raw. When the catalog ran and published reasoning-token paths, usage.reasoningTokens is the first path that yields a number. A path that yields nothing removes a runtime-supplied reasoningTokens value.

response.reasoningResolution is present when reasoningEffort was set or an explicit control bypassed the catalog:

{
  requestedEffort: "max",
  target: "openrouter",
  requestedModel: "~google/gemini-flash-latest",
  effectiveModel: "~google/gemini-flash-latest",
  outcome: "degraded",
  bypassed: false
}

The same record is on compile() output, stream.start, and stream.done.

The rest of the response is the runtime shape: id, status (completed, failed, requires_action, policy_violation), apiMode, model, messages, citations, images, patches, toolUsage, finishReason, warnings, errors, raw, and metadata. Direct OpenAI sets apiMode to "responses" and returns function calls as requires_action without running them. Cloudflare Responses does the same. Cloudflare chat, messages, and run report apiMode: "chat".

Streaming emits stream.reasoning.delta for thought text. That text is not copied into stream.text.delta, tool-argument deltas, or structured-output text. If a future catalog asks for think-tag stripping, tags are split out of answer deltas before those deltas are yielded.

The dispatcher does not store conversation history. When a directive lists history fields to drop, callers pass those paths to stripReasoningHistoryFields(message, paths). The helper clones the message, deletes only those pointers, and is safe to call twice. Catalog 5.1.0 paths are empty, so a real directive deletes nothing.

Streaming

for await (const event of dispatcher.executeStreamingChat(request)) {
  if (event.type === "stream.text.delta") process.stdout.write(event.data.text);
  if (event.type === "stream.reasoning.delta") { /* thought text */ }
}

Events, each with type, requestId, and data:

| Event | data | | --- | --- | | stream.start | model, apiMode, entrypoint, and reasoningResolution when present | | stream.text.delta | { text } | | stream.reasoning.delta | { text } | | stream.tool_call.delta | index, optional id, name, argumentsDelta | | stream.usage | { usage } | | stream.warning | { warning } | | stream.error | { error } | | stream.done | text, optional finishReason, model, optional reasoningText, optional reasoningResolution |

Tools

Local function tools work on all four providers. OpenRouter, Bedrock, direct OpenAI, and Cloudflare chat, responses, and messages run the tool loop when every returned call has an executor on the tool or in that provider's tools registry. The loop defaults are 8 iterations and 20 function calls. On direct OpenAI and Cloudflare, a call with no executor stops the loop: run() returns requires_action and does not send another request. Send that tool result back as a role: "tool" message with toolCallId. Streaming stays one request and does not run executors. Exceeding maxFunctionToolCalls fails the run with FUNCTION_TOOL_LIMIT. Cloudflare /ai/run forwards tools inside input and continues the loop only when the result is a chat completion with tool_calls.

OpenRouter server tools are webSearch, webFetch, datetime, imageGeneration, applyPatch, fusion, advisor, and subagent. Each policy has a mode of disabled, allowed, or required. Bedrock, direct OpenAI, and Cloudflare throw UNSUPPORTED_REQUEST_FIELD when any of them is enabled, and also reject a server_tool tool choice. A policy whose mode is disabled is not enabled.

Nested OpenRouter tools

Fusion, advisor, and subagent policies can set reasoning or reasoningEffort. They are not separate dispatcher calls. Before compilation, and only for OpenRouter, the dispatcher settles each policy once:

1. A child reasoning object stays.
2. Otherwise a child reasoningEffort is resolved with target openrouter and that tool's model.
3. Otherwise a parent reasoningEffort is resolved the same way for advisor model, subagent model, and fusion judgeModel.
4. Otherwise, when the parent supplied only an explicit reasoning object, the OpenRouter runtime copies that object onto nested tools that omit their own.
5. A nested tool with no model id is not looked up and does not inherit the parent wire body.

Each nested model is resolved on its own. A parent Bedrock or OpenAI body is never reused as another target's thinking config. A parent reasoningEffort also stops the OpenRouter runtime from copying the parent reasoning object onto those nested tools.

Prompt caching

cache is optional. A request that omits it compiles to the same provider body as before. Caching is resolved in the dispatcher from a local model profile. @x12i/ai-profiles is not called, and an unknown model still runs: the outcome is ignored and no marker is written.

explicit places one breakpoint at the end of the stable prefix. That prefix is system or instructions. The alert, user prompt, or other trailing turn stays after the breakpoint, so it can change without invalidating the prefix. There is no breakpoint when that prefix is missing. auto does not place a breakpoint. On GPT-5.6, GPT-6, Sol, and Luna it sends implicit cache options and prompt_cache_key. On Claude, Nova, and Qwen it does not cache unless you set mode to explicit. disabled removes cache markers that arrived through rawOpenRouterOverrides and writes none of its own.

key is sent as prompt_cache_key only for the OpenAI dialect. On other dialects it stays off the wire. cacheResolution.keyPresent is still true when you passed one. ttl is folded to the nearest value that model accepts. Folding sets cacheResolution.outcome to degraded and adds a CACHE_TTL_DEGRADED warning. The call still runs.

On Cloudflare, cache is also AI Gateway cache. disabled sends cf-aig-skip-cache: true and the outcome is applied. key is cf-aig-cache-key. 5m, 30m, and 1h are 300, 1800, and 3600 on cf-aig-cache-ttl. A model with no native prompt-cache profile still gets those headers and outcome applied. GPT on the Responses endpoint and Claude on the Messages endpoint also get the native breakpoint for that profile. Gateway headers are still sent. A per-request cloudflare.cacheTtlSeconds, cacheKey, or skipCache wins over the policy headers.

await dispatcher.run({
  model: "gpt-6-sol",
  system: playbook,
  prompt: alertPayload,
  cache: { mode: "explicit", key: "triage-playbook-v3", ttl: "30m" }
});

compile() returns cacheResolution and the warnings without sending the request. run() copies the same record onto the response. When the provider reports them, usage.cachedTokens and usage.cacheWriteTokens are set from OpenRouter prompt_tokens_details, OpenAI input_tokens_details, or Bedrock cacheReadInputTokens and cacheWriteInputTokens. Those fields are also filled when the provider cached implicitly and the request had no cache object.

| Target | Models | Explicit marker | Default TTL | Allowed TTL | | --- | --- | --- | --- | --- | | OpenAI | gpt-5.6, gpt-6, Sol, Luna | prompt_cache_breakpoint on the prefix, plus prompt_cache_options | 30m | 30m | | OpenAI | earlier GPT models | none; caching is automatic | n/a | n/a | | OpenRouter | openai/gpt-5.6, openai/gpt-6, Sol, Luna | same OpenAI marker on the chat or responses prefix | 30m | 30m | | OpenRouter | Claude | cache_control: { type: "ephemeral", ttl } on the last system block | 5m | 5m, 1h | | OpenRouter | Qwen | cache_control on the last system block | 5m | 5m | | OpenRouter | Gemini, DeepSeek, Grok, Moonshot, earlier GPT | none; caching is automatic | n/a | n/a | | Bedrock Converse | Claude, Nova | { cachePoint: { type: "default", ttl } } after system | 5m | 5m, 1h | | Bedrock Converse | OpenAI model ids | none. Those models cache through the Responses API, which this Converse runtime does not send | n/a | n/a |

TTL folding:

| Requested | GPT-5.6 / GPT-6 / Sol / Luna | Claude and Nova | Qwen | | --- | --- | --- | --- | | omitted | 30m | 5m | 5m | | 5m | 30m, degraded | 5m | 5m | | 30m | 30m | 5m, degraded | 5m, degraded | | 1h | 30m, degraded | 1h | 5m, degraded |

30m on Bedrock becomes 5m because 30 minutes is closer to 5 minutes than to 1 hour.

Warnings, not thrown errors:

| Code | When | | --- | --- | | CACHE_TTL_DEGRADED | The requested TTL was folded to a supported value and that value was sent. | | CACHE_UNSUPPORTED | The model has no explicit marker, auto was set on a model that only caches with a breakpoint, or the request has no stable prefix. | | CACHE_DIALECT_UNSUPPORTED | An OpenAI model id was sent to Bedrock Converse. |

Providers ignore a prefix under their minimum size (often 1,024 tokens) and cap how many breakpoints a request may carry. This dispatcher writes one breakpoint and does not count tokens.

A cache write can cost more than an uncached input. The first hit pays for the write when (write multiplier - 1) / (1 - read multiplier) is under 1. Later hits are the savings.

| Profile | Cache read | Cache write | Reads that pay for one write | | --- | --- | --- | --- | | Claude or Qwen, 5-minute TTL | 0.1x input | 1.25x input | 1 | | Claude, 1-hour TTL | 0.1x input | 2x input | 2 | | GPT-5.6, GPT-6, Sol, Luna | 0.25x to 0.5x input | 1.25x input | 1 | | Gemini, Grok, Moonshot, OpenAI before GPT-5.6 | about 0.25x to 0.5x input | no write premium | 1 |

Reuse the same key and the same prefix bytes. A changed playbook needs a new key, or the next call pays for another write.

Escape hatches

Set the provider control and leave reasoningEffort unset when you already have a wire object:

await dispatcher.run({
  provider: "openrouter",
  model: "openai/gpt-5.2",
  prompt: "Hello",
  reasoning: { effort: "low" }
});

await dispatcher.run({
  provider: "bedrock",
  model: "anthropic.claude-fable-5-1",
  prompt: "Hello",
  bedrockControls: {
    additionalModelRequestFields: {
      thinking: { type: "adaptive" },
      output_config: { effort: "low" }
    }
  }
});

rawOpenRouterOverrides still merges onto an OpenRouter body. Its metadata object is folded into metadata on every provider. Any other override key is rejected on Bedrock, direct OpenAI, and Cloudflare.

Supported features

| Feature | OpenRouter | Bedrock | OpenAI Responses | | --- | --- | --- | --- | | run, compile, executeStreamingChat | yes | yes | yes | | reasoningEffort via the catalog | yes | yes | yes | | cache explicit breakpoint | GPT-5.6 / GPT-6 family; Claude and Qwen via cache_control. Other models stay implicit | Claude and Nova via cachePoint. OpenAI ids are not marked | GPT-5.6 / GPT-6 family, including Sol and Luna. Earlier models stay implicit | | Explicit reasoning control | reasoning | bedrockControls.additionalModelRequestFields | reasoning | | Local function tools | yes, including the tool loop | yes, including the tool loop | yes, when every returned call has an executor; otherwise requires_action | | MCP tools from mcp.servers | run and compile | run and compile | run and compile. Streaming does not receive them | | OpenRouter server tools (fusion, advisor, subagent, datetime, and the rest) | yes | UNSUPPORTED_REQUEST_FIELD when enabled | UNSUPPORTED_REQUEST_FIELD when enabled | | responseFormat | forwarded | UNSUPPORTED_REQUEST_FIELD | placed on Responses text | | apiMode: "chat", rawOpenRouterOverrides | OpenRouter fields | UNSUPPORTED_REQUEST_FIELD | UNSUPPORTED_REQUEST_FIELD | | File content parts | forwarded | non-text parts throw UNSUPPORTED_CONTENT, except data-URL images | input_file |

Cloudflare supports the same run, compile, and executeStreamingChat entrypoints. executeStreamingChat rejects /ai/run. reasoningEffort uses the OpenAI or OpenRouter catalog, as described above. cache sets AI Gateway headers and, for GPT on Responses and Claude on Messages, the native breakpoint. Local function tools and MCP tools follow the OpenAI loop. OpenRouter server tools are rejected. rawOpenRouterOverrides is rejected except for a metadata object, which is folded into metadata. responseFormat is text on Responses, response_format on chat, and rejected on messages and run.

Credentials

A provider is available when the dispatcher can resolve the credential it sends. compile and run throw when a required value is missing.

| Provider | Option | Environment | | --- | --- | --- | | OpenRouter | openrouter.apiKey | OPENROUTER_API_KEY, then OPEN_ROUTER_KEY | | Bedrock | bedrock.region and optional bedrock.credentials | AWS_REGION or AWS_DEFAULT_REGION. When credentials is omitted, the AWS SDK default chain is used. An explicit credential is accessKeyId, secretAccessKey, and optional sessionToken. | | OpenAI | openai.apiKey | OPENAI_API_KEY | | Cloudflare | cloudflare.apiToken and cloudflare.accountId | CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID |

Errors

Catalog failures throw AiDispatcherError with code, message, and the catalog details. Dispatcher codes include:

| Code | When | | --- | --- | | PROVIDER_NOT_IMPLEMENTED | Provider is not openrouter, bedrock, openai, or cloudflare | | PROVIDER_REQUIRED | No provider string is available | | PROVIDER_ADAPTER_MISSING | An implemented provider has no adapter | | REASONING_MODEL_REQUIRED | reasoningEffort is set and no model id is available | | REASONING_PREFIX_UNSUPPORTED | A catalog prompt prefix has no user text to attach to | | UNSUPPORTED_REQUEST_FIELD | A field cannot be sent on the selected provider | | MODEL_REQUIRED | Direct OpenAI or Cloudflare has no model id | | INPUT_REQUIRED | Direct OpenAI or Cloudflare has no messages, input, or prompt | | OPENAI_API_KEY_MISSING | Direct OpenAI has no key | | CLOUDFLARE_API_TOKEN_MISSING | Cloudflare has no API token | | CLOUDFLARE_ACCOUNT_ID_MISSING | Cloudflare has no account id | | PROVIDER_AUTH_FAILED | OpenAI or Cloudflare HTTP 401 or 403 | | PROVIDER_MODEL_NOT_FOUND | OpenAI or Cloudflare HTTP 404 | | PROVIDER_REQUEST_FAILED | Other OpenAI or Cloudflare HTTP failures | | PROVIDER_RATE_LIMITED | OpenAI or Cloudflare HTTP 429 after retries | | PROVIDER_RETRYABLE / PROVIDER_GATEWAY_UNREACHABLE | OpenAI or Cloudflare transport failures after retries, including timeout | | FUNCTION_TOOL_LIMIT | Function-tool calls exceed maxFunctionToolCalls | | UNSUPPORTED_CONTENT | Bedrock received an image URL or content part it cannot send | | MCP_SERVER_INVALID | An MCP server is missing an id, a tools array, or callTool, or a server id is repeated | | MCP_TOOL_NAME_INVALID | An MCP server id or tool name uses characters other than letters, numbers, underscores, and hyphens, or the exposed name is longer than 64 characters | | MCP_TOOL_NAME_CONFLICT | Two MCP tools share an exposed name | | MCP_TOOL_NOT_FOUND | request.mcp names a tool that is not registered, or request.mcp is not false or a list of names | | MCP_LIST_FAILED | refreshMcp() failed, or the new list breaks the name rules. The previous catalog stays |

OpenAI HTTP failures such as 401 return a failed response from run(). FUNCTION_TOOL_LIMIT does too. Streaming yields stream.error and then throws, matching the other transports.

Logging

Pass logger on the dispatcher. run and executeStreamingChat emit ai-dispatcher.request.started at info. compile emits ai-dispatcher.request.compiled at debug. The record can include provider, entrypoint, streaming, requestedModel, effectiveModel, requestId, outcome, and bypassed. It does not include credentials, prompts, reasoning text, or raw bodies.

Limitations

• Catalog 5.1.0 does not currently emit a prompt prefix, a replacement model, think-tag stripping, history deletions, or a reasoning token budget. Those paths exist for the directive type and are covered by synthetic tests.
• Prompt caching is a dispatcher profile, not an @x12i/ai-profiles field. It writes one breakpoint after system or instructions. It does not place extra breakpoints on tools or on a prefix that changes in the middle of the call.
• Provenance notes about output-token headroom are not enforced, because the resolver does not return them as fields.
• OpenRouter server tools, apiMode: "chat", and rawOpenRouterOverrides stay on OpenRouter. Bedrock, direct OpenAI, and Cloudflare reject the server-tool fields and rawOpenRouterOverrides keys other than metadata. Cloudflare accepts apiMode: "chat" as the chat-completions endpoint.
• Direct OpenAI and Cloudflare streaming do not run the function-tool loop. MCP tools are not attached to executeStreamingChat on any provider. Cloudflare /ai/run does not stream.
• OpenRouter advisor, subagent, and fusion cannot call MCP sessions registered on the dispatcher.
• Bedrock run() returns a failed response for UNSUPPORTED_CONTENT. compile() throws that error.
• Live provider calls are not part of the package test suite. Tests mock HTTP and the Bedrock client and use the real 5.1.0 catalog for contract cases.