@cldmv/llm-translator
v0.13.0
Published
Bidirectional Anthropic Messages ⇄ OpenAI Chat Completions translator with backend auto-detection (vLLM and other OpenAI-compatible servers), built on slothlet, with an optional standalone server for running Claude Code against local models.
Maintainers
Readme
@cldmv/llm-translator
Bidirectional Anthropic Messages ⇄ OpenAI Chat Completions translator built on slothlet. It detects the backend behind an upstream (vLLM first, then any OpenAI-compatible server), converts requests, responses and streams in both directions, and ships an optional standalone server and CLI so Claude Code can run against local models.
Use it as a library when you want the conversion without any I/O (request, response and stream take plain objects), as a full round trip (send), or as a server that speaks the Anthropic or the OpenAI protocol to your clients.
Point Claude Code at a model running on your own GPU.
[!NOTE] Status: 0.x. Validated end to end for Claude Code (print mode) through the translator to vLLM. Other backends, model families and clients are covered by unit tests only and are tracked in the Real-backend validation epic. The API may change between minor versions.
✨ What's New
Latest: v0.13.0 (November 2026)
- OpenAI Responses API, both ways — clients such as Codex CLI can call
/v1/responses(stateless), and an upstream can be reached over its own Responses API, with encrypted reasoning carried between turns and a fall back to Chat Completions when the server has no/v1/responses. - One
apisetting,autoby default — each upstream picks its protocol withapi(messages/claude,chat/chatgpt,responses/codex,native/ollamaorauto). Underautoa request uses the client's own protocol when the server speaks it, so Claude Code → vLLM 0.11.1+ now goes to/v1/messageswith no setting. Setapi: "chat"to keep the old Chat Completions default;nativeMessagesis deprecated. - View full v0.13.0 Changelog
Recent Releases
- v0.12.0 (November 2026) — Prometheus
/metrics, HTTPS, per-key rate limits and CORS for the built-in server, and failed tool results marked by default over Chat Completions (Changelog) - v0.11.0 (November 2026) — native constrained decoding (JSON Schema, regex, choice, grammar),
clear_tool_usescontext edits,cache_saltandprompt_cache_keyhints, and host-first binding for hosted APIs (Changelog) - v0.10.0 (November 2026) — native
/v1/messagesfor vLLM, readiness probes and LoRA adapters, Groq, Together, Fireworks, xAI and Gemini drivers, and Ollama's native/api/chat(Changelog) - v0.9.0 (November 2026) — Azure OpenAI, OpenRouter, Hugging Face TGI and LocalAI drivers, with Azure's
apiVersionanddeploymentupstream fields and more startup probes (Changelog)
📚 For complete version history and detailed release notes, see the docs/changelog/ folder.
🚀 Key Features
🔁 Both Directions
Anthropic clients (Claude Code, the Anthropic SDKs) talk to OpenAI-compatible servers, and OpenAI clients talk to Anthropic upstreams. Requests, responses, streams, tool calls, usage, stop reasons and errors are converted; same-protocol traffic is forwarded without conversion.
🔎 Backend Detection
Each upstream is probed once and bound to a driver. The vLLM driver maps the server version to the right settings, checks whether the chat template accepts mid-conversation system messages and whether the model accepts images. See docs/drivers.md.
🧠 Reasoning and Tools
Anthropic thinking maps to the server's reasoning request and reasoning comes back as thinking blocks. Tool calls are buffered into whole tool_use blocks by default, with an incremental mode available.
📄 Images and PDFs
Image blocks are converted, PDFs become per-page text plus page images when the model accepts them, and a text-only model is handled by a startup probe and a one-time retry without images.
🖥 Server and CLI
llm-translator --upstream http://gpu-box:8000 starts a server with API-key protection for non-loopback binds, a client-header allow-list, and in-band error delivery for streams that are already committed. See docs/server.md.
🛡 100% Test Coverage
Lines, statements, functions and branches are all at 100% and enforced on every CI run, with no coverage-ignore pragmas. A harness in tools/e2e/ drives a real vLLM and a real Claude Code; see docs/testing.md.
📦 Installation
Requirements
- Node.js 22.15 or newer. Node.js 26 is fastest: PDF.js (through
unpdf) uses newer built-ins such asPromise.try,Float16ArrayandMap.prototype.getOrInsert, which Node.js 22 and 24 run through PDF.js' own polyfills. npm warns on Node.js 22 becauseunpdf2 declares Node.js 24; text extraction and page rendering work on 22.15. - Developing the package needs Node.js 22.22 or newer (
devEnginesenforces it on install andnpm run) - Fastify 5, only for the server and the Fastify plugin (an optional peer dependency)
Install
npm install @cldmv/llm-translator
npm install fastify # only for the server and the Fastify plugin🚀 Quick Start
Library
import { createTranslator } from "@cldmv/llm-translator";
const api = await createTranslator({
front: "anthropic", // the protocol your client speaks
upstreams: ["http://gpu-box:8000"], // vLLM is detected automatically; api defaults to "auto"
aliases: { "claude-opus-5-5": "qwen3-coder" }
});
// Pure conversion, no I/O: build the upstream call yourself.
const call = await api.request({ model: "claude-opus-5-5", max_tokens: 256, messages: [{ role: "user", content: "hi" }] });
// call.url, call.headers, call.body ...
// api.response(upstreamJson, call.state) converts a reply back,
// api.stream(upstreamSseBody, call.state) turns a stream into Anthropic events.
// Or let the translator do the full round trip.
const reply = await api.send({ model: "qwen3-coder", max_tokens: 256, stream: true, messages: [{ role: "user", content: "hi" }] });
if (reply.stream) for await (const { event, data } of reply.stream) console.log(event, data);
// Where would a request go, and what did it cost? (see docs/server.md)
const { upstream, model } = api.route({ model: "claude-opus-5-5" }); // no I/O
const { input_tokens, output_tokens, reported, estimated } = await reply.usage; // or send(body, { onUsage }); `estimated` holds the guesses written into the client reply when the upstream sent no usage
// Fallbacks, failovers and warnings are events (see docs/events.md); logging is off unless you set `logs`.
const stop = api.events.on("fallback", ({ endpoint, status, reason }) => console.log(endpoint, status, reason));Server and Claude Code
npx llm-translator --upstream http://gpu-box:8000 --port 8787 --model claude-opus-5-5=qwen3-coderconst server = await api.server.start({ port: 8787 });
console.log(server.url);Then point Claude Code at it with ANTHROPIC_BASE_URL=http://127.0.0.1:8787. The full walkthrough, including the vLLM flags tool calling needs, is in docs/claude-code.md.
📚 Configuration
createTranslator(config) takes front, upstreams (URLs or objects with url, apiKey, backend, models, default and settings, plus apiVersion and deployment for Azure OpenAI, see docs/drivers.md, and api, the protocol to speak to the server: "auto" (the default) uses the client's own protocol when the server has it, so an Anthropic client reaches vLLM 0.11.1 and later over /v1/messages and a Responses client reaches a server's /v1/responses, with a fall back to Chat Completions; "chat", "messages", "responses" and "native" (Ollama's own /api/chat) pick one explicitly, see docs/drivers.md. The old default was Chat Completions: set api: "chat" to keep it), aliases (a model name, or { model, effort, thinking } to pin settings), efforts (the reasoning levels each model supports, see docs/settings.md), global settings, pool (failover and load balancing between upstreams that serve the same model, see docs/server.md), detect, logs (off by default; true prints warnings, a path appends JSON lines), events and onEvent (see docs/events.md), capture (off by default; a directory or a callback receives redacted request and response records for debugging, see docs/events.md) and a custom transport. silent: true is a deprecated alias for logs: false. The CLI reads the same keys from flags, LLM_TRANSLATOR_* environment variables and a JSON file, in that order of precedence.
Settings tune how the converters map one protocol onto the other (inlineSystem, images, imageFallback, toolStreaming, pingInterval and more); they are resolved per upstream from defaults, probed values, driver defaults and your overrides. The full table is in docs/settings.md.
📚 Documentation
Reference
Guides
- Claude Code against vLLM
- Testing and the real-backend harness
- Known limitations
- Feature inventory
- Design decisions
Repo
🔗 Links
- npm: @cldmv/llm-translator
- GitHub: CLDMV/llm-translator
- Issues: GitHub Issues
- Releases: GitHub Releases
📄 License
Apache-2.0 © Shinrai / CLDMV
