free-ai-pool

Self-hosted AI load balancer and OpenAI-compatible proxy. Rotates across free-tier provider keys (Groq, Gemini, OpenRouter, DeepSeek, Cerebras, Mistral) with automatic failover on rate limits.

Quick start

Prerequisites: Node.js 18+. No separate database server — data is stored in SQLite at ~/.free-ai-pool/data.sqlite.

npx free-ai-pool
npx free-ai-pool logs    # dashboard URL + admin token (printed automatically)

Dashboard:  http://localhost:47821/?token=<admin-token>
Gateway:    http://localhost:47821/chat

CLI

| Command | Description | |---------|-------------| | npx free-ai-pool | Start in background (PM2) | | npx free-ai-pool logs | Show logs + dashboard URL | | npx free-ai-pool status | PM2 status | | npx free-ai-pool restart | Restart | | npx free-ai-pool stop | Stop | | npx free-ai-pool run | Foreground (debug) |

Persist across reboot (optional): pm2 startup then pm2 save

Setup

1. Open dashboard (?token= from logs)
2. Providers — add upstream API keys (use Gemini for PDF/image uploads)
3. Routing — choose how keys are picked (round robin, priority, etc.)
4. Client keys — create a key (fap_…) for your apps
5. Call POST /chat (see below)

Provider routing

The pool picks which upstream key handles each request, then failovers to the next key on rate limits (429), quota errors (402), or other retryable failures.

Configure in the dashboard Routing tab, or override per client key / per request.

| Strategy | Behavior | |----------|----------| | Round robin (default) | Spread load evenly using a per-request hash | | Priority / failover | Lower priority number first; others as backup | | Random | Random primary key each request | | Provider order | Try providers in configured order (e.g. Gemini → Groq → OpenRouter) | | Weighted random | Random primary weighted by each key’s weight (0 = never primary) | | Sticky | Same client key tends to hit the same provider key |

Per-key fields (Providers tab):

• priority — used by priority/failover and as tiebreaker (lower = sooner)
• weight — used by weighted random

File request routing (Routing tab):

• balanced (default) — route across all providers; non-file providers use fileText fallback
• strict_file — when a file is attached, try file-capable providers first
• Per-request override header: X-Free-AI-Pool-File-Routing: balanced|strict_file

Override order:

1. Request header X-Free-AI-Pool-Strategy: priority (optional, for testing)
2. Client key routing override (Client keys tab)
3. Pool default (Routing tab)

Response headers:

• X-Free-AI-Pool-Strategy — strategy used
• X-Free-AI-Pool-Provider — provider that handled the request
• X-Free-AI-Pool-Attempt — 1-based attempt index (1 = first key tried)
• X-Free-AI-Pool-File — attached | ignored | none (multipart)

PDF/image uploads still route to Gemini first when configured, regardless of strategy.

Authentication

| Audience | Method | |----------|--------| | Your apps (/chat) | Authorization: Bearer fap_<client-key> | | Dashboard / admin API | X-Admin-Token: <token> or ?token= in dashboard URL |

Client keys are created in the dashboard (Client keys tab). Shown once — copy immediately.

Do not expose client keys in browser frontend code. Call the pool from your backend.

Public API

Gateway routes require a client key. Rate limit: 120 req/min per key (configurable).

Call the pool from your backend — do not embed client keys in frontend JS.

GET /health

No auth. Returns { "status": "ok", "service": "free-ai-pool" }.

curl http://localhost:47821/health

POST /chat

OpenAI-compatible chat. Alias: POST /v1/chat/completions.

Auth: Authorization: Bearer fap_<key>

Simple chat (JSON)

curl http://localhost:47821/chat \
  -H "Authorization: Bearer fap_your_key" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}],"stream":false,"max_tokens":4096}'

const res = await fetch('http://localhost:47821/chat', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer fap_your_key',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    messages: [{ role: 'user', content: 'Hello' }],
    stream: false,
    max_tokens: 4096,
  }),
});
const data = await res.json();
console.log(data.choices[0].message.content);

import requests

resp = requests.post(
    "http://localhost:47821/chat",
    headers={
        "Authorization": "Bearer fap_your_key",
        "Content-Type": "application/json",
    },
    json={
        "messages": [{"role": "user", "content": "Hello"}],
        "stream": False,
        "max_tokens": 4096,
    },
)
print(resp.json()["choices"][0]["message"]["content"])

| Field | Type | Required | Notes | |-------|------|----------|-------| | messages | array | yes* | OpenAI chat messages | | prompt | string | yes* | Alternative to messages (single user instruction) | | stream | boolean | no | Default false | | model | string | no | Override model (provider default otherwise) | | max_tokens | number | no | Output token limit. OpenRouter defaults to 1024 if omitted — set higher for long replies |

*Either messages or prompt is required.

Tip: If replies cut off mid-answer, check finish_reason: "length" and raise max_tokens.

Document mode (multipart)

Send a PDF/image with a separate instruction. Do not dump parsed file text inside messages when also uploading file.

curl http://localhost:47821/chat \
  -H "Authorization: Bearer fap_your_key" \
  -F "[email protected]" \
  -F "prompt=Summarize this document in 4 bullets." \
  -F "fileText=<optional parsed text for non-file providers>" \
  -F "max_tokens=4096"

import fs from 'node:fs';

const form = new FormData();
form.append('file', new Blob([fs.readFileSync('document.pdf')]), 'document.pdf');
form.append('prompt', 'Summarize this document in 4 bullets.');
form.append('fileText', 'Optional parsed fallback for non-file providers');
form.append('max_tokens', '4096');

const res = await fetch('http://localhost:47821/chat', {
  method: 'POST',
  headers: { Authorization: 'Bearer fap_your_key' },
  body: form,
});
const data = await res.json();
console.log(data.choices[0].message.content);

| Field | Type | Required | Notes | |-------|------|----------|-------| | file | file | no | PDF or image (max 10MB) | | fileName | string | no | Original filename | | prompt | string | yes* | Instruction only (not the file dump) | | messages | JSON string | yes* | Simple chat history (instruction in last user message) | | fileText | string | no | Parsed text fallback when the provider cannot attach files | | stream | string | no | "true" or "false" | | model | string | no | Model override | | max_tokens | string/number | no | Same as JSON; OpenRouter defaults to 1024 if omitted |

*Provide prompt or messages.

Routing: File-capable providers (e.g. Gemini) get prompt/messages + file. Others get prompt/messages + fileText when fileText is set.

Response: Standard OpenAI chat completion JSON (streaming when stream: true).

Response headers: X-Free-AI-Pool-Strategy, X-Free-AI-Pool-Provider, X-Free-AI-Pool-Attempt, X-Free-AI-Pool-File (attached | ignored | none).

Environment variables

| Variable | Default | Description | |----------|---------|-------------| | PORT | 47821 | HTTP port (auto-increments if busy) | | FAP_DATA_DIR | ~/.free-ai-pool | SQLite database directory | | FAP_LOG_RETENTION_DAYS | 30 | Delete request logs older than this (0 = keep all) | | FAP_PUBLIC_HOST | auto | Override public URL in startup banner | | FAP_SKIP_PUBLIC_IP | — | Set 1 to skip public IP lookup | | FAP_GATEWAY_PATH | /chat | Custom primary chat path | | FAP_BIND_HOST | 0.0.0.0 | Bind address | | FAP_GATEWAY_RATE_LIMIT | 120 | Requests/min per client key | | OPENROUTER_HTTP_REFERER | http://localhost:47821 | OpenRouter referer header | | OPENROUTER_APP_TITLE | free-ai-pool | OpenRouter app title | | DEFAULT_COOLDOWN_MS | 60000 | Cooldown after 429/402 |

Local development

npm install
npm run build:ui
npm start              # PM2 background
npm run run            # foreground
npm run test:api       # integration tests (needs KEY + optional FILE)

# Test against running server
export KEY=fap_your_key
node test.mjs http://localhost:47821 "$KEY" ./resume.pdf

Admin API (dashboard only)

Used by the web UI. Not for app integration.

| Method | Path | Description | |--------|------|-------------| | GET | /api/admin/status | Pool status + 24h usage summary | | GET/PATCH | /api/admin/settings | Pool routing strategy + provider order | | GET | /api/admin/providers/meta | Provider catalog | | GET/POST/PATCH/DELETE | /api/admin/provider-keys | Manage upstream keys (priority, weight) | | GET/POST/PATCH/DELETE | /api/admin/client-keys | Manage client keys (optional routing override) | | GET | /api/admin/usage/summary?period=7d | Request/token totals (24h, 7d, 30d) | | GET | /api/admin/usage/by-provider?period=7d | Per-provider breakdown | | GET | /api/admin/usage/timeseries?days=7 | Daily request volume | | GET | /api/admin/usage/logs?limit=50 | Recent request log rows |

Auth: X-Admin-Token header (admin token from startup logs).

License

MIT