ai-api-check

Check your AI API setup for production reliability issues.

npx ai-api-check

No config. No API key. Scans your codebase and shows what can break when OpenAI, Claude, Gemini, or another AI provider slows down, rate limits, or goes offline.

ai-api-check looks for missing fallbacks, unsafe retries, unhandled 429s, missing timeouts, weak request tracing, and provider lock-in.

Why use this?

Most AI apps fail in the same few ways:

• one provider, no fallback
• no timeout protection
• no Retry-After handling
• retries without backoff or caps
• no request IDs or trace logs
• hardcoded provider URLs or models
• streaming calls that can hang
• no outage-readiness plan

Run a scan before these issues hit production.

Quick start

npx ai-api-check

Scan a specific directory:

npx ai-api-check ./src

Run focused checks:

npx ai-api-check --fallback
npx ai-api-check --timeout
npx ai-api-check --429
npx ai-api-check --retry
npx ai-api-check --request-id
npx ai-api-check --lockin
npx ai-api-check --hung
npx ai-api-check --outage
npx ai-api-check --sim

Example report

AI API Check — Reliability Report

Overall Score: 32/100  🚨 Production Risk

Scanned: 14 files  │  AI files: 3  │  Providers: openai

Checks:
❌ Fallback provider          single provider, no fallback
❌ Timeout protection         3/3 files unprotected
❌ 429 / rate-limit handling  no 429 handling detected
❌ Retry & backoff            3 file(s) missing retries
❌ Request ID & tracing       3/3 files missing request IDs
✅ Provider lock-in           no hardcoded models or URLs
✅ Hung request detection     no hung request risk
❌ Outage readiness           0/5 dimensions covered
❌ Fallback simulation        if openai fails → all requests fail

Run focused checks:
npx ai-api-check --fallback
npx ai-api-check --timeout
npx ai-api-check --429
npx ai-api-check --retry

Checks

| Check | Flag | What it finds | | ------------------------- | -------------- | ------------------------------------------------------------------------------------ | | Fallback provider | --fallback | Single-provider dependency, no backup route, no failover path | | Timeout protection | --timeout | AI calls missing AbortController, AbortSignal.timeout, or request timeout guards | | 429 / rate-limit handling | --429 | Missing 429 checks, no Retry-After handling, unsafe retry behavior | | Retry & backoff | --retry | No retries, infinite retries, or retries without exponential backoff and jitter | | Request ID & tracing | --request-id | AI calls with no request ID, trace ID, latency logging, or correlation metadata | | Provider lock-in | --lockin | Hardcoded model names, provider URLs, or missing baseURL override | | Hung requests | --hung | AI calls that can stall indefinitely, especially in UI/serverless paths | | Outage readiness | --outage | 5-part readiness check: fallback, timeout, retry, logging, and circuit breaker | | Fallback simulation | --sim | Simulates primary provider failure and shows which calls would fail |

Focused checks

Fallback provider

npx ai-api-check --fallback

Finds apps that depend on one AI provider with no backup route.

Good for answering:

If OpenAI goes down, does my app still work?

Timeout protection

npx ai-api-check --timeout

Finds AI calls that can hang forever because they have no timeout guard.

Good for catching:

stuck requests
frozen UI
serverless timeout risk
background jobs that never finish

429 / rate-limit handling

npx ai-api-check --429

Finds missing or unsafe handling for rate limits and quota errors.

Checks for:

429 handling
Retry-After handling
retry caps
retry storm risk
quota exceeded handling

Retry & backoff

npx ai-api-check --retry

Finds missing retry logic or retry logic that can make failures worse.

Checks for:

exponential backoff
jitter
max retry count
safe transient-error handling

Request ID & tracing

npx ai-api-check --request-id

Finds AI calls that are hard to debug after they fail.

Checks for:

request IDs
trace IDs
latency logs
provider metadata
route metadata
error classification

Provider lock-in

npx ai-api-check --lockin

Finds hardcoded provider assumptions.

Checks for:

hardcoded model names
hardcoded API URLs
missing baseURL override
single-provider SDK usage

Hung request detection

npx ai-api-check --hung

Finds calls that can freeze UI flows, API routes, or background workers.

Outage readiness

npx ai-api-check --outage

Scores whether your app is ready for provider degradation or outages.

Checks:

fallback
timeout protection
safe retries
request tracing
circuit breaker / failure guard

Fallback simulation

npx ai-api-check --sim

Simulates your primary provider failing and reports what happens next.

Example:

Primary provider: openai
Fallback provider: none
Result: all detected AI calls fail if openai is unavailable

Scoring

Each scan returns an overall reliability score from 0 to 100.

| Score | Status | | ------ | ------------------ | | 80–100 | ✅ Reliable | | 60–79 | ⚠️ Needs Work | | 40–59 | 🔶 At Risk | | 0–39 | 🚨 Production Risk |

Supported providers

Detects usage of:

• OpenAI
• Anthropic / Claude
• Google Gemini
• Azure OpenAI
• Cohere
• Mistral
• Together AI
• Fireworks AI
• Replicate
• DeepSeek
• OpenRouter
• LiteLLM
• OpenAI-compatible clients

Supported files

Scans:

.js
.ts
.jsx
.tsx
.mjs
.cjs
.py

Skips:

node_modules
.git
dist
build
.next
venv
__pycache__

CI usage

Run ai-api-check in CI to catch reliability regressions before they ship.

npx ai-api-check --ci

Example GitHub Actions step:

- name: Check AI API reliability
  run: npx ai-api-check --ci

Use CI mode to fail builds when production-risk issues are found (overall score below 40).

Common issues this catches

ai-api-check helps detect code patterns that lead to:

• OpenAI 429 errors
• Claude / Anthropic timeout issues
• Gemini quota exceeded errors
• OpenRouter provider failures
• stuck streaming responses
• retry storms
• serverless function timeouts
• unexpected AI API bills
• hard-to-debug failed requests
• provider outage failures

When to use ai-api-check vs ai-doctor

Use ai-api-check when you want a production-readiness scan:

npx ai-api-check

Use ai-doctor when you already have a specific broken AI API issue and want debugging/fix help:

npx ai-doctor

Simple rule:

ai-api-check = check reliability before production
ai-doctor = debug/fix a broken AI API issue

Want managed retries, fallback routing, and request tracking?

ai-api-check shows what can break.

AI Badgr helps you run production AI requests with:

• retries and backoff
• fallback routing
• OpenAI-compatible API
• request IDs
• latency and route metadata
• provider backup policies

Get an API key at:

https://aibadgr.com