LLMTrace

Zero-Code LLM API Traffic Inspector & Analytics Platform

LLMTrace is a transparent proxy that captures, visualizes, and analyzes LLM API traffic — zero SDK, zero code changes required. Simply point your OPENAI_BASE_URL (or equivalent) at the proxy endpoint and get full visibility into your LLM usage, token consumption, and costs.

✨ Features

• 🔌 Zero Integration — No SDK to install, no code to change. Just update your baseURL.
• 📊 Real-time Visualization — Live request/response viewer with streaming support.
• 💰 Cost Estimation — Automatic token tracking and cost calculation for 9+ models (GPT-4o, Claude 3, etc.).
• 🔍 Token Analytics — Per-request, per-model, and per-environment token usage breakdown.
• 🛡️ API Key Pass-through — Your keys never touch the proxy. Raw Authorization headers are forwarded as-is.
• 💾 Multiple Storage Backends — Memory (ephemeral), SQLite (local), MySQL / PostgreSQL (team/shared).
• 🚀 Multiple Deployment Modes — Standalone server, Whistle plugin, or shared team service.

🚀 Quick Start

1. Install

# Global install
npm i -g @smartyi/llmtrace

# Or run on demand (no install)
npx @smartyi/llmtrace

2. Run

# Defaults: standalone, port 8080, in-memory storage, no auth
llmtrace

# Custom port + password protection
llmtrace -p 3000 -P my-secret

# Persistent SQLite
llmtrace --db ./llmtrace.db

# Shared (multi-tenant) mode with MySQL
llmtrace --mode shared --mysql-url mysql://user:pw@host:3306/llmtrace

# Show all options
llmtrace -h

CLI flags

| Flag | Default | Description | |------|---------|-------------| | -m, --mode <mode> | standalone | standalone or shared | | -p, --port <port> | 8080 | Listening port | | -H, --host <host> | 0.0.0.0 | Bind address | | -P, --password <pw> | (none) | Standalone simple-password auth | | --db <path> | :memory: | SQLite path (standalone) | | --mysql-url <url> | — | MySQL DSN (shared mode) | | --admin-mysql-url <url> | =--mysql-url | Admin DB DSN (shared) | | --jwt-secret <secret> | — | JWT signing secret (shared) | | -c, --config <path> | — | Path to config file | | -h, --help | — | Show help | | -v, --version | — | Show version |

Configuration file

Drop a .llmtracerc.json (or llmtrace.config.js / package.json#llmtrace) next to your project; it will be picked up automatically:

{
  "port": 3000,
  "password": "my-secret",
  "db": "./llmtrace.db"
}

Precedence: CLI flags > config file > environment variables > built-in defaults.

Environment variables (legacy compat)

| Variable | Equivalent flag | |----------|-----------------| | PORT | --port | | LLMTRACE_DB | --db | | LLMTRACE_PASSWORD | --password | | LLMTRACE_MYSQL_URL | --mysql-url | | LLMTRACE_ADMIN_MYSQL_URL | --admin-mysql-url | | JWT_SECRET | --jwt-secret | | LLMTRACE_HOST | --host | | LLMTRACE_MODE | --mode | | LLMTRACE_WEB_ROOT | --web-root |

3. Point Your LLM Client at the Proxy

Change your client's base URL to use the proxy route format:

# Original
OPENAI_BASE_URL=https://api.openai.com/v1

# With LLMTrace
OPENAI_BASE_URL=http://localhost:8080/proxy/my_env/https/api.openai.com/v1

That's it. Every request is now captured, token usage is tracked, and costs are calculated.

4. View Your Traffic

Open http://localhost:8080 in your browser to see the visualization panel.

5. One-shot setup for OpenClaw gateway (Linux)

If you run OpenClaw as a systemd --user service and want its outbound LLM traffic visible inside LLMTrace, run:

npx @smartyi/whistle.llmtrace openclaw

The interactive installer will:

1. Install whistle globally (npm i -g whistle) if missing.
2. Start whistle on 127.0.0.1:8899.
3. Install the LLMTrace plugin (w2 install @smartyi/whistle.llmtrace).
4. Download the whistle root CA to ~/.WhistleAppData/whistle_rootCA.cer.
5. Locate the OpenClaw service file via openclaw gateway status.
6. Inject http_proxy / https_proxy / no_proxy / NODE_USE_ENV_PROXY / NODE_EXTRA_CA_CERTS into a managed block of that service file (a .bak.<ts> backup is kept).
7. systemctl --user daemon-reload.
8. Restart OpenClaw.
9. Print an SSH tunnel command so you can open the whistle UI from your laptop: ssh -N -L 8899:127.0.0.1:8899 root@<host>.

The installer is idempotent and only touches a clearly-marked managed block; re-running it is safe.

Prefer doing it by hand? See docs/openclaw-setup.md for the equivalent step-by-step guide.

6. Admin Panel (Shared Mode)

In shared mode, the Admin SPA is served at /#/admin/ and the backend REST API is available at /admin/* backed by MySQL.

Bootstrap the initial system administrator once:

LLMTRACE_ADMIN_MYSQL_URL=mysql://user:pass@host:3306/llmtrace \
[email protected] \
LLMTRACE_BOOTSTRAP_PASSWORD='change-me' \
node packages/llmtrace/dist/modes/shared/bootstrap.js

Then open http://localhost:8080/#/admin/ in your browser to log in. You can also drive the API directly at POST /admin/login to obtain a JWT. Major endpoints:

| Area | Endpoints | |------|-----------| | Auth | POST /admin/login, POST /admin/refresh, GET /admin/me | | Teams | GET/POST /admin/teams, GET/PATCH/DELETE /admin/teams/:id, POST /admin/teams/:id/restore | | Members | GET/POST /admin/teams/:id/users, PATCH/DELETE /admin/teams/:id/users/:uid, POST /admin/teams/:id/users/:uid/restore | | Global users | GET /admin/users, PATCH /admin/users/:id/system-admin (system admin only) | | Environments | GET/POST /admin/environments, GET/PATCH/DELETE /admin/environments/:id, POST /admin/environments/:id/restore |

All DELETE operations are soft deletes: rows are tombstoned with a deleted_at timestamp and hidden from default listings. Append ?includeDeleted=1 to list endpoints to see tombstones, and call the matching /restore endpoint to revive a resource (as long as its natural key — slug / email / envId — is still free).

📐 Proxy URL Format

All proxy requests follow a consistent, human-readable path structure:

/proxy/{env_id}/{scheme}/{host}/{path}

| Segment | Description | Example | |---------|-------------|---------| | env_id | Environment/tenant identifier | prod, dev, team_a | | scheme | Upstream protocol | https, http | | host | Upstream hostname (+ port) | api.openai.com, localhost:11434 | | path | Full API path | /v1/chat/completions |

Examples

# OpenAI Chat Completions
curl http://localhost:8080/proxy/prod/https/api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

# Local Ollama
curl http://localhost:8080/proxy/local/http/localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3", "messages": [{"role": "user", "content": "Hi"}]}'

# Anthropic
curl http://localhost:8080/proxy/prod/https/api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_KEY" \
  -d '{"model": "claude-3-sonnet-20240229", ...}'

🔌 API Reference

Compatibility API (Viz Panel)

| Endpoint | Method | Description | |----------|--------|-------------| | /cgi-bin/requests | GET | List all captured requests | | /cgi-bin/request/:id | GET | Get single request detail | | /cgi-bin/clear | POST | Clear all captured data |

Analytics API

| Endpoint | Method | Description | |----------|--------|-------------| | /api/stats | GET | Aggregated usage statistics | | /api/timeseries | GET | Time-series data for charting | | /api/cost-estimate | GET | Single-request cost estimation |

Stats Response

{
  "totalRequests": 42,
  "totalPromptTokens": 15000,
  "totalCompletionTokens": 8500,
  "totalTokens": 23500,
  "estimatedCost": 0.127,
  "avgResponseTimeMs": 845,
  "modelBreakdown": [
    { "model": "gpt-4o", "requests": 30, "totalTokens": 18000, "estimatedCost": 0.095 },
    { "model": "gpt-4o-mini", "requests": 12, "totalTokens": 5500, "estimatedCost": 0.032 }
  ]
}

Cost Estimate

curl "http://localhost:8080/api/cost-estimate?promptTokens=1000&completionTokens=500&model=gpt-4o"
# → {"promptTokens": 1000, "completionTokens": 500, "model": "gpt-4o", "estimatedCost": 0.0075}

⚙️ Configuration

Deployment Modes & Storage

LLMTrace ships three deployment modes, each with a dedicated storage strategy:

| Mode | Request Store | Admin Store | Entry | |------|---------------|-------------|-------| | Standalone | Memory or SQLite | — | llmtrace (default) | | Whistle Plugin | In-memory (capped 10k) | — | w2 install @smartyi/whistle.llmtrace | | Shared (team) | MySQL | MySQL (same or separate DSN) | llmtrace --mode shared --mysql-url ... |

Configuration sources & precedence

For the llmtrace CLI, configuration is resolved from (highest precedence first):

1. CLI flags (-p, -P, --db, --mysql-url, …)
2. Config file picked up via cosmiconfig:
   • .llmtracerc / .llmtracerc.{json,yaml,yml,js,cjs,mjs}
   • llmtrace.config.{js,cjs,mjs,json}
   • package.json#llmtrace
3. Environment variables (see table below)
4. Built-in defaults

Use -c <path> to point at an explicit config file.

Environment Variables

| Variable | Equivalent flag | Applies to | Default | Description | |----------|-----------------|------------|---------|-------------| | PORT | --port | all | 8080 | Listening port | | LLMTRACE_HOST | --host | all | 0.0.0.0 | Bind address | | LLMTRACE_MODE | --mode | all | standalone | standalone or shared | | LLMTRACE_DB | --db | standalone | :memory: | SQLite path, or :memory: for ephemeral | | LLMTRACE_PASSWORD | --password | standalone | (none) | Simple-password protection (empty = open) | | LLMTRACE_MYSQL_URL | --mysql-url | shared | (required) | MySQL DSN for the request store | | LLMTRACE_ADMIN_MYSQL_URL | --admin-mysql-url | shared | =LLMTRACE_MYSQL_URL | Admin DB DSN (can be a separate database) | | JWT_SECRET | --jwt-secret | shared | dev fallback | JWT signing secret — must be set in production | | LLMTRACE_WEB_ROOT | --web-root | all | (auto) | Override the static SPA directory |

Supported Models (Cost Estimation)

| Model | Input ($/1M tokens) | Output ($/1M tokens) | |-------|---------------------|----------------------| | gpt-4o | $2.50 | $10.00 | | gpt-4o-mini | $0.15 | $0.60 | | gpt-4 | $30.00 | $60.00 | | gpt-3.5-turbo | $0.50 | $1.50 | | claude-3-opus | $15.00 | $75.00 | | claude-3-sonnet | $3.00 | $15.00 | | claude-3-haiku | $0.25 | $1.25 |

🏗️ Architecture

┌─────────────┐      ┌───────────────────────────────────┐      ┌─────────────────┐
│  LLM Client │ ────► │         LLMTrace Proxy            │ ────► │  Upstream API   │
│  (OpenAI SDK│       │                                   │       │  (api.openai.   │
│   or curl)  │       │  ┌─────────────┐  ┌────────────┐ │       │   com, etc.)    │
│             │ ◄──── │  │ Body Buffer │  │ SSE Parser │ │ ◄──── │                 │
└─────────────┘       │  └──────┬──────┘  └─────┬──────┘ │       └─────────────────┘
                      │         │                │        │
                      │  ┌──────▼────────────────▼──────┐ │       ┌─────────────────┐
                      │  │        Token Extractor       │ │       │   Viz Panel     │
                      │  └──────────────┬───────────────┘ │       │   (SPA)         │
                      │                 │                 │ ◄──── │                 │
                      │  ┌──────────────▼───────────────┐ │       └─────────────────┘
                      │  │         Request Store        │ │
                      │  │  (Memory / SQLite / MySQL/PG)│ │
                      │  └──────────────────────────────┘ │
                      └───────────────────────────────────┘

Core Modules

| Module | Path | Description | |--------|------|-------------| | router-parser | src/core/router-parser.ts | Parses /proxy/{env_id}/{scheme}/{host}/{path} URLs | | parser | src/core/parser.ts | Request/response JSON parsing, summary extraction | | sse-stream | src/core/sse-stream.ts | SSE event stream parsing, content & tool_call aggregation | | token-extractor | src/core/token-extractor.ts | Token count extraction from JSON or SSE streams | | analytics | src/analytics/index.ts | Usage aggregation, cost estimation, time series | | store | src/core/store.ts | Abstract RequestStore interface |

Deployment Modes

| Mode | Entry Point | Storage | Use Case | |------|-------------|---------|----------| | Standalone | src/modes/standalone/ | Memory / SQLite | Single-user, local dev | | Whistle Plugin | src/modes/whistle/ | Memory (capped) | Browser proxy workflows | | Shared Service | src/modes/shared/ | MySQL | Multi-tenant team deployment with /admin API |

🧪 Development

# Install dependencies (project uses pnpm workspaces)
pnpm install

# Build everything (backend packages + frontend SPA)
pnpm run build

# Run tests
pnpm test -- --run

# Watch mode for the standalone server (TS watcher + Vite HMR + node --watch)
pnpm dev:standalone

# Lint & format
pnpm run lint
pnpm run format

Project Structure

llmtrace/
├── src/
│   ├── core/              # Core parsing & extraction logic
│   │   ├── types.ts       # TypeScript interfaces
│   │   ├── parser.ts      # Request/response parsing
│   │   ├── router-parser.ts
│   │   ├── sse-stream.ts
│   │   ├── token-extractor.ts
│   │   └── store.ts       # Store interface
│   ├── stores/            # Storage implementations
│   │   ├── memory-store.ts
│   │   ├── sqlite-store.ts
│   │   └── mysql-store.ts
│   ├── modes/             # Deployment mode entry points
│   │   ├── standalone/    # Memory / SQLite, single-user
│   │   ├── whistle/       # Whistle plugin adapter (in-memory)
│   │   └── shared/        # Multi-tenant team mode (MySQL)
│   ├── analytics/         # Usage stats & cost estimation
│   ├── admin/             # Multi-tenant admin (auth, users, envs, alerts)
│   ├── frontend/          # Viz React+Vite source (builds to src/web/viz)
│   └── web/viz/           # Built SPA assets served by koa-static
├── test/                  # Vitest unit tests
├── docs/                  # Architecture & planning docs
└── package.json

Frontend Development

The Viz panel is a React 18 + Vite SPA located at src/frontend/. Development workflow:

# One-time: install frontend deps
pnpm run install:frontend

# Terminal 1 — backend with auto-reload (port 8080)
pnpm dev:standalone

# Terminal 2 — frontend dev server with HMR (port 3000, proxies /cgi-bin, /api, /admin to 8080)
pnpm dev:frontend

# Or just run dev:standalone above — it concurrently starts the
# TypeScript watcher, the Vite dev server, and a `node --watch` server.

# Production build (backend + frontend bundle into src/web/viz)
pnpm run build

📋 Roadmap

• [x] Core parsing engine (parser, SSE, tokens)
• [x] Standalone server with proxy routing (Memory / SQLite)
• [x] Whistle plugin — uiServer + resStatsServer, in-memory store
• [x] Shared mode — MySQL-backed, team/user/environment model
• [x] Analytics API (stats, cost, time-series) with env/model/time filters
• [x] Admin API — teams, users, environments CRUD with role-based auth
• [x] Viz panel — React + Vite SPA with real-time streaming preview
• [x] Admin dashboard UI — Dashboard, Environments, Members, Teams pages
• [x] Invite flow — email-based token invite with SHA-256 hash storage
• [x] Soft deletes — all resources support tombstoning and restore
• [ ] Export reports (CSV, JSON)
• [ ] GitHub Actions CI workflow
• [ ] Docker Compose setup

📄 License

MIT