@wenathlan/gateway

The universal AI gateway library — any LLM, any base URL, any API key.

Current release: 1.1.13 — see CHANGELOG.md and SECURITY.md for the supported release line.

Features

• 12 auth methods: bearer, apikeyheader, queryparam, basic, oauth2clientcredentials, jwtsign, sigv4, hmacsign, cookie, mtls, keylesssdk, anonymous
• 7 routes per version (auto-generated): chat/completions, completions, messages, responses, embeddings, keys, models
• Unlimited versions: create v1 through v9 and beyond
• Model rotation: persession or perrequest, every N messages
• Base URL rotation: round-robin or on-failure upstream switching
• Retry with backoff: exponential + cryptographically secure jitter, configurable retryable statuses
• 429 handling: retry, rotate model, rotate base URL or fallback — per status class
• Cross-provider fallback: opencode-kilo style chain
• 2-calls thinking pattern: zai SDK style with fresh response guarantee
• SSE anti-parse streaming: brace-depth parser, heartbeat discard, broken-chunk recovery, model masking
• Context sharing: optional session history restoration + truncation, shared or isolated
• Meta-models: compose multiple individual models behind one dispatch id
• Universal DB: Prisma 7 + LibSQL adapter (works on Vercel, Netlify, any Node host)
• Spring Boot intelligent CLI: scaffold everything interactively
• No platform Functions: standalone Node process — runs anywhere

Repository layout

gateway/
  index.ts          # library entry — createGateway, createServer, types
  engine.ts         # universal engine — routes, rotation, retry, fallback
  http.ts           # the http transport — hono server + SSE pipeline (one file)
  cli.ts            # interactive CLI (init/add/keys/serve/export)
  configloader.ts   # config validation + web/config.ts loading
  authentication.ts # 12 auth methods + key resolution
  database.ts       # prisma + libsql client
  utils.ts          # shared helpers (secure randomness, ids, headers)
  types.ts          # public types
  web/              # YOUR design + customization (deploy-ready interface)
    app.tsx         # web console (self-contained, no main.tsx)
    config.ts       # your version definitions (data only)
    schema.prisma   # your database schema
    globals.css
    index.html      # vite entry
    vite.config.ts  # builds into ../dist
    capacitor.config.ts # android wrapper — same interface
    vercel.json, netlify.toml # platform manifests
  Dockerfile        # the one container file (node 26 slim, non-root —
                    # compose merged INTO it: ENV/EXPOSE/VOLUME/HEALTHCHECK
                    # + the hardened docker run line)
  pom.xml, settings.xml          # maven adapter (io.github.wenathlan:gateway)
  gateway.csproj, GatewayCli.cs  # nuget adapter (Gateway)
  gateway.gemspec                # rubygems adapter (gateway)
  Gateway.java                   # java adapter
  .github/workflows/ # release + publish + security + maintenance suite

The root is the universal library (dry, no hardcoding). Everything project-specific lives in web/ as data.

Quick start

# scaffold a new gateway
npx @wenathlan/gateway init

# add a version
npx @wenathlan/gateway add v6

# register keys
npx @wenathlan/gateway keys v6

# push the database (prisma 7: the datasource url lives in the
# scaffolded prisma.config.ts, the client generates into node_modules)
npm i -D prisma
npx prisma db push && npx prisma generate

# validate the configuration
npx @wenathlan/gateway validate

# serve locally
npx @wenathlan/gateway serve

Library mode

import { createGateway } from "@wenathlan/gateway";

const gateway = createGateway({
  versions: [
    {
      id: "v1",
      routes: ["chat/completions", "completions", "messages", "responses", "embeddings", "keys", "models"],
      auth: { method: "bearer", token: () => process.env.MY_TOKEN },
      models: [
        { id: "gpt-x", upstream: "gpt-x", context: 128000, maxoutput: 16384 },
      ],
      rotation: { mode: "persession", everyN: 12, models: ["gpt-x"] },
      retry: { max: 3, backoffbasems: 400, backoffcapms: 8000 },
      upstreams: [{ baseurl: "https://api.example.com/v1" }],
    },
  ],
});

Container

The one container file (stencil standard — a container serves one thing well; the compose orchestration is merged INTO the Dockerfile):

# the library image
docker build -t gateway/library:1.1.13 .

# run the stack (every former compose setting as run flags)
docker run -d --name gateway-core --restart unless-stopped --init \
  --cap-drop ALL --security-opt no-new-privileges:true \
  --ulimit nofile=65536:65536 --memory-swap -1 --shm-size 2g \
  -p 31080:3001 -v gatewaydata:/data gateway/library:1.1.13

The server listens on PORT (default 3001). The state lives in the gatewaydata volume (/data/gateway.db); the build cache is the inline cache embedded in the published GHCR image (there is no separate buildcache image).

Publish chain

Bumping package.json + adding the matching CHANGELOG.md section and pushing to main automatically:

1. creates the v{version} tag,
2. builds the zip -9 source archive + container bundle + SHA-256 checksums,
3. publishes the GitHub release with the curated notes,
4. publishes npmjs, GitHub Packages npm, Maven, NuGet, RubyGems and GHCR (the library image, multi-arch).

Every root metadata file (pom.xml, gateway.csproj, gateway.gemspec, the Dockerfile GATEWAY_VERSION default, CHANGELOG section) must carry the package version — the CI envelopes gate enforces the lockstep.

License

MIT — see LICENSE.