n8n-nodes-multi-model

Multi-Model Chat Model — a production-grade n8n Chat Model node that chains multiple LLM providers (OpenRouter, OpenAI-compatible, Google Gemini) behind a single connection to the n8n AI Agent, with automatic failover, retries with exponential backoff, per-model circuit breakers, and health-aware routing — all inside the same Agent execution. No new Agent run, no lost conversation state, no lost tool calls when a model fails mid-execution.

AI Agent
   │
   └── Chat Model
          │
          ▼
Multi-Model Chat Model
          │
          ├── deepseek/deepseek-v4-flash   (priority 1)
          ├── google/gemini-3-flash        (priority 2)
          └── qwen/...                     (priority 3)

Installation

From npm (once published)

In n8n: Settings → Community Nodes → Install → n8n-nodes-multi-model

From source

npm install
npm run build
npm test

Then copy (or symlink) the package into your n8n custom-nodes folder, or npm link it into your self-hosted n8n instance, and restart n8n.

Configuration

1. Credentials

Create a Multi-Model API credential:

| Field | Purpose | | -------- | --------------------------------------------------------------------------------- | | API Key | Default key used by any model row that doesn't set its own API Key override | | Base URL | Default Base URL for OpenAI-Compatible rows without their own override |

This credential is encrypted at rest by n8n and is never written into the workflow JSON or logs — it's the recommended path if all (or most) of your models share one OpenRouter key.

Credential Isolation: if a specific model needs a different key (e.g. DeepSeek via OpenRouter Key A, Gemini via a separate Gemini key), set the API Key (Override) / Base URL (Override) fields directly on that model's row in the node UI. These are password-masked in the editor but — unlike the credential — are stored with the workflow definition, so prefer the shared credential wherever possible and only use per-row overrides when you genuinely need multiple keys.

2. Connect to the AI Agent

Drag Multi-Model Chat Model onto the canvas, connect its output to the Chat Model input of an AI Agent, Basic LLM Chain, or any node that accepts a LangChain Chat Model. The Agent sees exactly one Chat Model — it has no idea multiple providers are involved.

3. Add Models

Click Add Model to add each provider in priority order (top = tried first):

• Provider: OpenRouter / OpenAI-Compatible / Google Gemini
• Model ID: the provider's own identifier, e.g. deepseek/deepseek-v4-flash, google/gemini-3-flash, gpt-4.1, gemini-2.5-flash — never hardcoded, always whatever the provider currently publishes
• Max Retries (This Model): same-model retry budget before moving to the next model (default 1)
• Optional per-1M-token pricing for cost-tracking metadata

Reorder, enable/disable, or remove rows freely — the list is fully dynamic (1, 2, 5, 10+ models).

4. Options / Failover Conditions / Retry & Backoff / Circuit Breaker / Timeouts / Advanced

Each is its own collapsible section so the default view stays simple:

• Options — temperature, max tokens, top-p, penalties, stop sequences (applied to whichever model ends up serving the request; unsupported params are simply omitted per provider rather than causing a hard failure)
• Failover Conditions — checkboxes for exactly which error types trigger a fallover to the next model (timeout, network error, 429, 5xx, empty response, invalid response/ JSON, tool-call error, provider unavailable, or "Any Error" to override everything)
• Retry & Backoff — initial delay / multiplier / max delay (defaults: 1000ms / ×2 / 5000ms)
• Circuit Breaker — failure threshold, tracking window, cooldown (defaults: 5 failures / 60s / 60s)
• Timeouts — per-request timeout and the maximum total time the node is allowed to spend across all attempts and fallbacks combined (defaults: 30s / 60s)
• Advanced — logging verbosity (Off / Errors Only / Normal / Debug — never logs API keys, headers, or credentials, only masked previews) and whether to allow OpenAI-Compatible Base URLs pointing at private/internal networks (off by default, an SSRF guard for self-hosted "Base URL" providers like a local vLLM server)

How failover works

Try Model #1 (priority order)
   ↓ success → return result, same session/messages/tools untouched
   ↓ failure (matches a checked Failover Condition)
Try Model #2
   ↓ success → return result
   ↓ failure
Try Model #3
   ↓ ...
All configured models exhausted
   ↓
Throw MultiModelFailoverError with full per-model attempt detail
(no fabricated response — the workflow handles it via an Error Trigger)

Same-model retries (with exponential backoff) happen before moving to the next model, and only for error types that are retryable by default: timeout, network error, 429, 5xx, empty response, invalid response. Auth errors (401/403) and bad requests are never retried on the same model — they go straight to the next model in the list (which may use a different credential).

Nothing about the conversation changes on failover. The exact same messages array — including system prompt, prior tool calls, and tool results — is re-sent to the next model. The AI Agent's loop, memory, and tool state are completely unaffected; only the underlying provider changes.

Health-aware routing & circuit breaker

Each model has its own circuit breaker:

• CLOSED — requests flow normally.
• OPEN — after Failure Threshold failures within Failure Window, the model is marked unhealthy and is skipped entirely (no request attempted) until Cooldown elapses. New requests go straight to the next healthy model instead of repeatedly hammering a dead endpoint.
• HALF_OPEN — after cooldown, exactly one probe request is allowed through. Success closes the circuit (model returns to normal rotation); failure re-opens it for another full cooldown.

Health state currently lives in an in-memory store scoped per node instance (per workflow

• node name) within the running worker process — this is enough for most self-hosted setups. If you run n8n in queue mode with multiple workers and need globally consistent health state across all of them, the storage layer (core/HealthManager.ts) is a plain interface (get/set); implement a RedisHealthStore against it and the rest of the system (circuit breaker, failover manager) needs no changes.

Tool calling & structured output across failover

Tools, tool results, and their IDs are passed through unmodified between the Agent and whichever model is currently active. If the primary model supports tool calling but a lower-priority fallback doesn't, that fallback is skipped automatically (not tried and failed) — the next compatible model is used instead. The same applies to structured output / JSON mode.

Cost & usage metadata

Every call is reported to n8n's own execution log the same way a native Chat Model sub-node is — click the green checkmark on the node after a run to see exactly which model answered, the full failover attempt trail, and token usage, without any of it being injected into the chat content the Agent/LLM itself sees:

{
  "model": "openRouter:google/gemini-3-flash",
  "primaryModel": "openRouter:deepseek/deepseek-v4-flash",
  "fallbackUsed": true,
  "response": "...",
  "tokenUsage": { "promptTokens": 512, "completionTokens": 128, "totalTokens": 640 },
  "finishReason": "stop",
  "latencyMs": 2460,
  "attempts": [
    { "modelId": "...", "providerModel": "...", "errorType": "TIMEOUT", "succeeded": false, "retryCount": 1, "latencyMs": 1820 },
    { "modelId": "...", "providerModel": "...", "succeeded": true, "retryCount": 0, "latencyMs": 640 }
  ]
}

If real token usage is available from the provider, it's used as-is; if not, a token count is estimated and explicitly flagged (usage.isEstimate: true) rather than silently guessed as if it were exact.

Providers

| Provider | Notes | | --- | --- | | OpenRouter | First-class. Point Model ID at whatever OpenRouter currently publishes (deepseek/..., google/..., qwen/..., etc.) — nothing is hardcoded. | | OpenAI-Compatible | Any server speaking the OpenAI Chat Completions wire format: OpenAI itself, Together, Fireworks, vLLM, LM Studio, a self-hosted proxy. Set a custom Base URL. Private/internal Base URLs (localhost, 10.x, 192.168.x, cloud metadata IPs) are blocked by default — see Advanced → Allow Private Network Base URLs if you're intentionally pointing at a local server. | | Google Gemini | Native Gemini API wire format (not OpenAI-compatible) — handled by a dedicated adapter. |

Adding a new provider means adding one adapter class implementing ChatModelProvider (core/types.ts) plus one case in core/ProviderManager.ts — nothing else in the system needs to change.

Error codes

| Type | Retried on same model? | Fails over by default? | | --- | --- | --- | | TIMEOUT | Yes | Yes | | NETWORK_ERROR | Yes | Yes | | RATE_LIMIT (429) | Yes | Yes | | SERVER_ERROR (5xx) | Yes | Yes | | EMPTY_RESPONSE | Yes | Yes | | INVALID_RESPONSE | Yes | Yes | | TOOL_ERROR | No | Configurable | | STRUCTURED_OUTPUT_ERROR | No | Configurable | | AUTH_ERROR (401/403) | No | Always (next model may use a different key) | | BAD_REQUEST (400/404/422) | No | Yes (may be model-specific misconfiguration) | | UNSUPPORTED_FEATURE | No | Yes (skip to a compatible model) |

If every configured model fails, the node throws a structured MultiModelFailoverError (never a fabricated response) with the full list of attempts and their error types, so it can be caught by an Error Trigger node or a Try/Catch-style branch in your workflow. Raw provider error bodies (e.g. "OpenRouter 502 Bad Gateway") are kept in internal logs only — customer-facing content never includes them directly.

Example workflow

See examples/example-workflow.json:

Webhook → AI Agent → Multi-Model Chat Model (DeepSeek → Gemini → Qwen) + Google Sheets Tool → Respond

Import it via Workflows → Import from File, add your credentials, and adjust model IDs.

Troubleshooting

• "Add at least one model..." — the Models list is empty; add at least one row.
• "...has no API Key" — set the node credential's API Key, or an API Key Override on that specific row.
• "Invalid Base URL...targets a private/internal network" — you pointed an OpenAI-Compatible row at localhost/a private IP. If intentional (local vLLM/LM Studio), enable Allow Private Network Base URLs under Advanced.
• All models keep failing with UNSUPPORTED_FEATURE — the Agent is using tools or structured output that none of your configured models support; add at least one model known to support tool calling.
• Circuit breaker seems "stuck" — it stays OPEN until Cooldown elapses; lower Failure Threshold/Cooldown for faster recovery during testing, or disable the circuit breaker entirely under Advanced while debugging.

Production recommendations

• Put Failure Threshold: 3–5, Cooldown: 30–60s for latency-sensitive chat use cases; raise the cooldown for expensive/rate-limited providers.
• Set Maximum Total Failover Time below your upstream caller's own timeout (e.g. a WhatsApp webhook) so the workflow always responds before the caller gives up.
• Keep at least one fallback model on a different provider account/region than the primary, so a provider-wide outage doesn't take down every model at once.
• Use Logging: Errors Only in production; switch to Debug only while diagnosing an issue, since Debug logs every attempt (still with credentials masked).

Development

npm install
npm run build     # tsc + copy node icon into dist/
npm test           # Jest unit tests: ErrorClassifier, RetryManager, CircuitBreaker, FailoverManager
npm run lint

Architecture

nodes/MultiModelChatModel/
  MultiModelChatModel.node.ts   n8n node definition + UI + supplyData()
  FailoverChatModel.ts                  LangChain BaseChatModel wrapper the Agent talks to

core/
  FailoverManager.ts     orchestrates priority + health-aware routing across models
  RetryManager.ts         exponential backoff, per-model retry budget
  CircuitBreaker.ts       closed/open/half-open per model
  ErrorClassifier.ts      raw errors -> normalized types -> retry/failover decisions
  HealthManager.ts        pluggable health store (in-memory now, Redis-ready interface)
  CostTracker.ts          usage-based or configured-pricing cost estimation
  ProviderManager.ts      builds a provider adapter from a model's config row
  types.ts                shared interfaces (ChatModelProvider, ModelConfig, etc.)

providers/
  BaseHttpAdapter.ts   shared OpenAI-wire-format request/response handling
  OpenRouterAdapter.ts
  OpenAIAdapter.ts      generic OpenAI-compatible (custom Base URL, SSRF-guarded)
  GeminiAdapter.ts       native Gemini wire format

utils/
  logging.ts      leveled logger, all output passed through masking
  masking.ts       redacts anything that looks like a credential before logging
  validation.ts    Base URL / SSRF checks, blank-content and JSON helpers

Security

• API keys are stored via n8n's encrypted credential system by default; per-row overrides are password-masked in the editor.
• Logs never contain API keys, Authorization headers, or other credential-shaped fields — everything passed to the logger is redacted first.
• Custom Base URLs (OpenAI-Compatible provider) are checked against a private/internal IP and hostname blocklist (loopback, link-local, RFC1918 ranges, cloud metadata endpoints) to reduce SSRF risk; this can be explicitly disabled per-workflow for legitimate local-server use cases.
• Raw upstream error bodies are kept out of the response returned to the calling workflow; they're available in internal logs (when logging is enabled) for debugging only.

This is a best-effort application-level guard, not a substitute for network-level egress controls on the host running n8n — pair it with a firewall/allowlist if this node is ever exposed to fully untrusted input.

License

MIT