BrowseComp CLI (TypeScript)

A command-line tool for evaluating LLM browser search capabilities via a function-calling agent loop. The agent autonomously uses web search (Brave Search API / DuckDuckGo), web content extraction, and local Python execution to gather evidence and answer queries.

TypeScript port of the Python browsecomp-cli, with identical CLI options, NDJSON event protocol, and tool behavior.

Installation

npm install -g browsecomp-cli

Or run from source:

npm install
npm run build
node dist/main.js --query "..."

Requirements

• Node.js >= 18
• An OpenAI-compatible LLM API endpoint
• (Optional) Brave Search API key for stable, higher-quality search (keyless DuckDuckGo is used otherwise)
• Local python3 binary available in PATH (for the Python tool)

Usage

browsecomp \
  --base-url https://api.openai.com/v1 \
  --api-key sk-your-key-here \
  --query "Find the 2024 Nobel Physics Prize winners and their contributions"

With Brave Search

browsecomp \
  --base-url https://api.openai.com/v1 \
  --api-key sk-your-key-here \
  --brave-key your-brave-key \
  --query "What are the latest developments in quantum computing?"

Using Environment Variables

export OPENAI_BASE_URL=https://api.openai.com/v1
export OPENAI_API_KEY=sk-your-key-here
export BRAVE_API_KEY=your-brave-key

browsecomp --query "Your evaluation query here"

CLI Options

| Option | Required | Default | Description | |--------|----------|---------|-------------| | --base-url <url> | Yes* | - | LLM API base URL | | --api-key <key> | Yes* | - | LLM API key | | --model <name> | No | gpt-4o | Model name | | --query <query> | Yes | - | Evaluation task query | | --brave-key <key> | No | - | Brave Search API key (optional; falls back to keyless DuckDuckGo) | | --max-turns <n> | No | 10 | Maximum agent loop iterations | | --no-stream | No | false | Disable streaming output | | -V, --version | - | - | Show version | | -h, --help | - | - | Show help |

*Can be set via environment variables OPENAI_BASE_URL and OPENAI_API_KEY

Output Format

The tool outputs NDJSON (Newline Delimited JSON). Each line is a self-contained JSON object:

{"type": "thinking", "content": "I need to search for..."}
{"type": "tool_call", "name": "brave_search", "arguments": {"query": "Nobel Physics 2024"}}
{"type": "tool_result", "name": "brave_search", "output": "{\"results\": [...]}", "success": true}
{"type": "answer", "content": "Explanation: ... Exact Answer: ... Confidence: 95%"}
{"type": "done", "total_turns": 3}

Event types: thinking, tool_call, tool_result, answer, error, done.

# Extract only the final answer
browsecomp --query "..." | jq 'select(.type == "answer") | .content'

Available Tools

duckduckgo_search

Search the web via DuckDuckGo. No API key required and enabled by default — the zero-config search backend. Falls back to the DuckDuckGo Lite endpoint when the primary HTML endpoint yields nothing, and finally to Baidu web search when DuckDuckGo rate-limits the client (202 anomaly page), so consecutive agent searches keep working.

brave_search

Search the web using the Brave Search API (independent index, official JSON API). More stable than scraping DuckDuckGo, with a monthly-refreshing free tier (2,000 queries/month, 1 query/sec). Requires --brave-key (or BRAVE_API_KEY). Only offered to the agent when a key is configured.

web_extract

Extract readable text content from a web page URL. Converts HTML to clean text (default max 8000 chars).

run_python

Execute Python code in a local subprocess (requires python3 in PATH).

Security Warning

⚠️ WARNING: The run_python tool executes arbitrary Python code on your local machine. Only use this tool in trusted, isolated evaluation environments.

Development

npm install
npm run build       # compile TypeScript to dist/
node dist/main.js --help

Publishing to npm

npm login
npm publish         # prepublishOnly runs the build automatically

If the package name is taken, publish under a scope:

npm publish --access public   # with "name": "@your-scope/browsecomp-cli"

Project Structure

src/
├── index.ts             # Package exports + version
├── main.ts              # CLI entry point (commander)
├── config.ts            # Configuration management
├── prompts.ts           # System prompt definitions
├── stream.ts            # NDJSON stream emitter
├── agent.ts             # Core agent loop
└── tools/
    ├── index.ts         # Tool registry
    ├── types.ts         # Shared tool typing
    ├── brave.ts         # Brave Search API integration
    ├── duckduckgo.ts    # DuckDuckGo search (keyless, default)
    ├── webExtract.ts    # Web content extraction
    └── pythonExec.ts    # Python subprocess execution

License

MIT