@bitslix/blxbench

BLXBench TUI/CLI: Benchmarks in the terminal (Ink + Bun).

Install (npm, native binary)

The published package ships one native executable per platform (Linux x64, macOS arm64/x64, Windows x64) via optional dependencies. You only need Node.js (20+) for the small bin launcher; the app itself is a standalone blxbench binary.

npm i -g @bitslix/blxbench
# or: pnpm add -g @bitslix/blxbench
blxbench --help

pnpm global installs use a different global layout than npm; the command above is still valid, but the global bin path is tool-specific.

Report browser (optional)

The desktop report viewer is a separate npm scope: @bitslix/blxbench-report-browser (metapackage + optional native Tauri binaries per OS, same idea as @bitslix/blxbench + platform packages). It is not installed with the CLI by default.

• TUI: /report browser install (uses npm under ~/.blxbench/lib/report-browser/), then /report browser open — only starts the Tauri app (no default system-browser flow). /report browser uninstall removes that npm-prefix tree; it does not delete copies under ~/.blxbench/bin or unset BLXBENCH_REPORT_BROWSER_BINARY (you manage those yourself).
• Override: set BLXBENCH_REPORT_BROWSER_BINARY to a blxbench-report-browser executable from your own build — when set, the TUI does not auto-sync the npm install to the CLI version.
• Auto-sync: after a global CLI upgrade (pnpm update -g @bitslix/blxbench, etc.), the TUI runs a one-time check: if the metapackage under ~/.blxbench/lib/report-browser is older than this blxbench build, it runs the same pinned npm install as /report browser install. Set BLXBENCH_REPORT_BROWSER_NO_AUTO_SYNC=1 (or true / yes) to disable. If npm is missing, sync is skipped (short log line when an upgrade would have been needed).

You need Node.js and npm on PATH for /report browser install. Headless servers need a GUI session for the desktop app (or use BLXBENCH_REPORT_BROWSER_BINARY on a machine with a display).

Playwright

The CLI is built with Playwright not embedded in the native binary (native Playwright is large and not fully bundleable with the current build). The npm package installs playwright as a normal runtime dependency, and the small Node launcher passes that package path to the native binary for render validation.

Audio (minigames)

Minigame music/SFX are played by spawning an external player process.

• Linux/macOS: install mpv (recommended) or ffmpeg (for ffplay).
• Windows: current builds use ffplay or mpv. (If you ship a bundled ffplay.exe in the Windows platform package, no extra install is needed.)

If no supported player is found, the minigames stay silent but the benchmark runner works normally.

Development (this monorepo)

Requires Bun 1.2+.

cd apps/blxbench-cli
pnpm install   # from repo root
bun run dev

Typecheck: pnpm run typecheck. Build JS bundle: pnpm run build.

Native release binaries (for npm or local testing)

From repo root, build all platform binaries (cross-compiled, typically from Linux/x64) into packages/blxbench-cli-*/:

pnpm --filter @bitslix/blxbench run release:all

Single-host smoke builds (artefacts under build/):

pnpm --filter @bitslix/blxbench run release:linux
# release:mac-arm | release:mac-x64 | release:windows

Environment & API keys

Variable precedence for each key (highest first):

1. Shell / exported process.env — never overridden by the CLI.
2. .env files — next to the installed CLI package, then cwd/.env (unless ignoreWorkspaceDotenv is set in config).
3. ~/.blxbench/config.json — optional env object fills missing keys.

A root .env in the project directory is optional. You can store keys in ~/.blxbench/config.json instead (the first-run wizard saves OPENROUTER_API_KEY there).

• preferStoredEnv (boolean in config, default false): when true, non-empty values under env in config override values from .env files (shell exports still win).
• ignoreWorkspaceDotenv (boolean): when true, cwd/.env is not loaded (CLI-adjacent .env still loads).
• BLXBENCH_PREFER_STORED_ENV: 1 / true / yes or 0 / false — overrides preferStoredEnv for that process without editing the JSON file.

When both cwd/.env and stored env in config define the same key (e.g. OPENROUTER_API_KEY) and neither preferStoredEnv nor ignoreWorkspaceDotenv resolves it, the TUI asks once per transition (including before the shell after login) whether to prefer the project file, the saved config, or to skip the workspace .env.

Desktop notifications (optional)

When a benchmark run finishes, the CLI can show a native desktop hint (macOS Notification Center, Linux notify-send, Windows tray balloon). Nothing is shown if the run was cancelled in the TUI or if the environment cannot display notifications.

• TUI: /set notify on or /notify on — stored in ~/.blxbench/config.json as desktopNotify. Omit the argument to toggle, or use on / off.
• Environment: BLXBENCH_NOTIFY=1 or true enables for that process; BLXBENCH_NOTIFY=0 or false forces off (e.g. CI) even when the config file has notifications enabled.
• Headless: --notify, or the same env/config rules.

The config file is merged into env at startup together with .env files (see Environment & API keys above). Session saves also live under ~/.blxbench/.

Deep links & launch args (TUI)

The leaderboard web app can offer links of the form:

blxbench://run?model=<model-id>

<model-id> is the full public model id (URL-encoded), e.g. openrouter/anthropic/claude-3.5-sonnet → blxbench://run?model=openrouter%2Fanthropic%2Fclaude-3.5-sonnet.

Optional query param: provider=<alias-or-adapter-id> (e.g. opr or openrouter). If omitted, the TUI picks the provider from the first path segment of model using the embedded adapter registry.

Invoking the CLI: your OS must pass that URL as an argument to blxbench (same as blxbench 'blxbench://run?model=...'). Without a registered URL handler, the browser cannot start the app automatically.

Without a custom scheme, you can preload the TUI from a terminal:

blxbench --provider opr --models openrouter/anthropic/claude-3.5-sonnet

(Interactive TTY only; headless still requires --headless / non-TTY as today.)

Registering blxbench:// on Linux (example)

Create ~/.local/share/applications/blxbench-uri.desktop (adjust the Exec path to your blxbench):

[Desktop Entry]
Name=BLXBench
Exec=/usr/local/bin/blxbench %u
Type=Application
NoTerminal=true
MimeType=x-scheme-handler/blxbench;

Then:

chmod +x ~/.local/share/applications/blxbench-uri.desktop
xdg-mime default blxbench-uri.desktop x-scheme-handler/blxbench

macOS / Windows

Opening blxbench:// links requires an app or helper that forwards the URL to the blxbench executable (e.g. a small wrapper or “Open URL” automation). The exact setup depends on how you installed the CLI.

License

MIT — see LICENSE.