@openstellar/tool-search

v1.1.0

Published

14 days ago

Tool search plugin for OpenCode — BM25 and regex search to discover tools on demand, reducing context usage

0High
0Medium
0Low

chewji9875

opencode plugin tool-search bm25 context-reduction deferred-loading

OpenStellar Tool Search

The 100-Tool Dilemma in Agentic Coding

Modern agentic engineering workflows connect multiple Model Context Protocol (MCP) servers: codebase knowledge graphs, git providers, issue trackers, database inspectors, browser automation, and terminal tools.

In a standard environment with ~100 MCP tools:

60,000+ tokens of static tool descriptions and parameter schemas are injected into every single prompt turn.
Context Window Saturation: Over 50% of the active context window is consumed before the agent reads a single line of your codebase.
Model Degradation & Hallucination: Heavy system prompts cause attention drift, leading the model to call outdated tools, guess parameter shapes, or ignore instructions.
Session Startup Deadlocks: OpenCode freezes session tool snapshots at boot (~0–2s). Slow-starting MCP servers get dropped or orphaned permanently.

The Solution: Deferred Tool Virtualization

OpenStellar Tool Search virtualizes tool delivery inside OpenCode (supporting both OpenCode 1.x and OpenCode 2.0 / opencode2 seamlessly):

Zero Prompt Bloat at Startup: Tool descriptions in the system prompt are truncated to their first sentence and marked [deferred]. Full parameter schemas are always preserved — parameters are never hidden or substituted, on both OpenCode 1.x and 2.0.
Local Hybrid Search Engine: Tools are indexed locally using BM25 Okapi and local ONNX vector embeddings (@xenova/transformers) running in a background Node.js worker thread.
On-Demand Tool Delivery: When an agent needs a tool, it calls tool_search (by natural language task) or tool_search_regex (by exact name or wildcard). Full tool descriptions and parameters are delivered dynamically.
OpenCode v2 Parallel MCP Prewarming: Enabled MCP servers start concurrently before returning hooks, guaranteeing all tools are safely captured in OpenCode's initial startup snapshot without unbounded hangs (fail-open timeouts).
Dual-Target OpenCode Compatibility: Native adapter exports support OpenCode 1.x plugin hooks and OpenCode 2.0 setup/transform lifecycles concurrently.

Demo in Action & How It Works

🎥 Video Demo:

https://github.com/user-attachments/assets/4cad5981-6f0c-42d8-b7c5-32f42d6c5c7b

┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│  AI AGENT RUNTIME: 288 tools loaded ([deferred])                                                │
│                                                                                                  │
│  Agent Intent: "Search the codebase knowledge graph for auth handlers"                          │
│                                                                                                  │
│  1. Semantic Discovery:                                                                          │
│     tool_search({ query: "find functions in code graph" })                                       │
│     └── Hits: [ codebase_memory_mcp_search_graph (score: 0.94), codebase_memory_mcp_trace_path ] │
│                                                                                                  │
│  2. Exact ID / Regex Discovery:                                                                  │
│     tool_search_regex({ pattern: "^codebase_memory_mcp_" })                                     │
│     └── Unlocks: codebase_memory_mcp_search_graph, codebase_memory_mcp_trace_path, etc.          │
│                                                                                                  │
│  3. Execution:                                                                                   │
│     codebase_memory_mcp_search_graph({ query: "auth handlers", project: "app" })                │
│     └── Output: [ src/auth/jwt.ts:handleAuth, src/auth/session.ts:verifySession ]                │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘

Installation

🤖 1-Click AI Setup (Recommended)

Paste this 1-line instruction into your OpenCode assistant to install, configure, and migrate your MCP servers automatically:

Please read https://raw.githubusercontent.com/open-stl/openstellar-tool-search/main/INSTALL_PROMPT.md and follow its instructions to install and configure @openstellar/tool-search for OpenCode.

👉 Want to inspect the prompt manually? See INSTALL_PROMPT.md.

Manual Setup

Install the package globally:

npm install -g @openstellar/tool-search

Add @openstellar/tool-search to your opencode.jsonc (or .opencode/opencode.json):

OpenCode 1.x (opencode):

{
  "plugin": [
    [
      "@openstellar/tool-search@latest",
      {
        "maxResults": 5,
        "mode": "hybrid"
      }
    ]
  ]
}

OpenCode 2.0 (opencode2):

{
  "plugins": [
    {
      "package": "@openstellar/tool-search@latest",
      "options": {
        "maxResults": 5,
        "mode": "hybrid"
      }
    }
  ]
}

(Note: OpenCode 2.0 also supports the array-tuple format ["@openstellar/tool-search@latest", { ... }] in "plugins" or "plugin" for seamless backward compatibility).

Restart OpenCode or opencode2. Your tools will automatically appear with [deferred] tags in the system prompt.

How Dual Search Works

1. Search by Intent (`tool_search`)

When the model knows what task it wants to achieve but doesn't know the exact tool identifier:

tool_search({ query: "create a pull request on GitHub" })

Performs Reciprocal Rank Fusion (RRF) combining BM25 keyword matching and vector cosine similarity.
Delivers the full description, canonical ID, and parameter documentation.

2. Search by Name / Pattern (`tool_search_regex`)

When the model or agent needs a specific known tool, namespace, or regex pattern:

// Exact lookup
tool_search_regex({ pattern: "^github_create_issue$" })

// Pattern / namespace sweep
tool_search_regex({ pattern: "^(read|write|edit|glob|grep|bash)$" })

3. Deferred Authorization Lifecycle

Search-Gated Invocation: Calling an unsearched [deferred] tool returns a helpful [Tool Search Required] message pointing to the canonical search.
Session Persistence: Authorizations are cached per session (persisted atomically in ~/.cache/openstellar/tool-search/auth-state.json with a 30-day TTL).
Compaction Sync: When Sleev or OpenCode compacts context history, tool authorizations are cleanly reset so stale tool assumptions do not pollute subsequent reasoning turns.

Empirical Context Reduction & Scientific Benchmark

🔬 Empirical Live Evaluation: Measured across real production MCP tools using the standard Compact Wire JSON format (the actual minified payload transmitted over HTTP to LLM APIs) with the official Xenova/gpt-4o BPE tokenizer (o200k_base).

💡 What the deferral removes: only verbose description prose (everything after the first sentence). Parameter schemas — the bulk of every tool definition — stay fully present in the tools array at all times. This is deliberate: parameters are what the model needs to construct correct calls, and keeping them intact guarantees zero behavioral drift between the deferred and authorized states.

┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ PROMPT CONTEXT FOOTPRINT COMPARISON (Compact Wire JSON Format)                                   │
├──────────────────────────────────────────────────────────────────────────────────────────────────┤
│                                                                                                  │
│  BASELINE (Static Prompt Injection):                                                             │
│  [████████████████████████████████████████████████████████████] ~47,430 – 67,000 tokens / turn   │
│                                                                                                  │
│  WITH TOOL SEARCH DESCRIPTION DEFERRAL:                                                          │
│  [███████████████████████████████████████████████████░░░░░░░░] ~44,784 – 49,755 tokens / turn   │
│  └── NET SAVED PER TURN: -2,646 to -17,200+ tokens (Scales with server documentation verbosity)  │
│                                                                                                  │
│  LIVE RUNTIME TELEMETRY RECONCILIATION:                                                          │
│  • OpenAI o200k_base (Compact Wire) : ~49,755 tokens (Tool array)                                │
│  • DeepSeek API (Ollama Cloud)      :  58,835 tokens (Total turn payload with messages)         │
│  • Google Gemini API (Antigravity)  :  54,986 tokens (Total turn payload with messages)         │
│                                                                                                  │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘

🌟 Design Philosophy — Description Deferral, Not Schema Hiding:
Descriptions are deferred: collapsed to a first-sentence [deferred] stub. The model discovers the full description on demand via tool_search / tool_search_regex.
Parameter schemas are never deferred: they remain byte-identical in every turn. Tool-calling accuracy is driven by the tools array — keeping it intact means calls are correct from the very first invocation after authorization.
Compact Wire Accounting: Measured on actual serialized wire payloads (JSON.stringify(tools)) without synthetic indentation or newline padding, ensuring reported numbers match proxy and gateway dashboards.

1. Token Reduction Across Production Tool Categories (Compact Wire JSON)

Measured with the Xenova/gpt-4o BPE tokenizer (o200k_base) across real production MCP tool schemas:

Top Single-Tool Description Savers:

codebase-memory-mcp_query_graph: Desc 407 → 27 tokens (−382 tokens saved, −68.7%)
context7_resolve-library-id: Desc 398 → 22 tokens (−376 tokens saved, −64.6%)
github-grep_searchGitHub: Desc 337 → 23 tokens (−314 tokens saved, −49.3%)
codebase-memory-mcp_search_graph: Desc 324 → 19 tokens (−305 tokens saved, −34.6%)
codebase-memory-mcp_index_repository: Desc 224 → 12 tokens (−212 tokens saved, −43.4%)
codebase-memory-mcp_index_status: Desc 200 → 96 tokens (−104 tokens saved, −36.4%)
codebase-memory-mcp_search_code: Desc 178 → 11 tokens (−167 tokens saved, −36.7%)
codebase-memory-mcp_trace_path: Desc 170 → 11 tokens (−159 tokens saved, −22.2%)
exa_web_search_exa: Desc 113 → 19 tokens (−94 tokens saved, −39.2%)
codebase-memory-mcp_get_code_snippet: Desc 104 → 14 tokens (−90 tokens saved, −48.6%)

2. Information Retrieval & Evaluation Metrics (TREC / BEIR / BFCL Protocol)

Evaluated across representative developer natural-language queries adhering to standard information retrieval benchmarks:

| Evaluation Metric | Score / Value | 95% Bootstrap Confidence Interval ($B=2,000$) | Benchmark Protocol Standard | | :--- | :---: | :---: | :--- | | NDCG@1 | 1.0000 | — | Single-Shot Graded Relevance | | NDCG@3 (Normalized Discounted Cumulative Gain) | 0.9421 | $[0.9173, 0.9669]$ | TREC / BEIR Graded Relevance ($r \in [0, 3]$) | | NDCG@5 | 0.9421 | — | Top-5 Graded Ranking Fidelity | | MRR (Mean Reciprocal Rank) | 1.0000 | $[1.0000, 1.0000]$ | First Relevant Tool Reciprocal Rank | | MAP (Mean Average Precision) | 1.0000 | $[1.0000, 1.0000]$ | Multi-Tool Composition Precision Rank | | Hit Rate@1 (Top-1 Accuracy) | 100.0% | — | Single-Shot Exact Discovery | | Hit Rate@3 (Top-3 Accuracy) | 100.0% | — | Guaranteed Discovery within Top-3 | | Hit Rate@5 (Top-5 Accuracy) | 100.0% | — | Full Discovery Coverage | | Search Latency (p50 / p95 / p99) | 0.066 ms / 0.099 ms / 0.099 ms | — | In-Memory BM25 + ONNX Embedding Worker |

xychart-beta
    title "Search Latency vs. LLM Turn Generation Time (ms)"
    x-axis ["Tool Search (p50)", "Tool Search (p99)", "IPC / MCP Wire", "Local LLM TTFT", "Cloud LLM TTFT"]
    y-axis "Response Time (ms)" 0 --> 1200
    bar [0.066, 0.099, 2.5, 350, 1200]

⚡ Zero Perceptible Overhead: At 0.066 ms, tool search latency represents $<0.006%$ of typical cloud LLM Time-to-First-Token (TTFT), running over 18,000× faster than model inference.

3. Multi-Turn Compounding Scale & Cost Savings

Because system prompt tool definitions are re-transmitted on every single conversational turn, savings compound linearly ($\mathcal{O}(T)$) throughout an agent session ($T = 1\text{–}100\text{ turns}$, standard rate: $$2.50\text{ / 1M input tokens}$):

xychart-beta
    title "Multi-Turn Compounding Cumulative Token Savings across 288 Tools (k Tokens)"
    x-axis ["T=1", "T=5", "T=10", "T=20", "T=30", "T=50", "T=100"]
    y-axis "Cumulative Tokens Saved (k Tokens)" 0 --> 2200
    line [21.1, 105.7, 211.5, 422.9, 634.4, 1057.4, 2114.8]

| Session Horizon ($T$) | Cumulative Baseline Tokens | With Tool Search | Net Tokens Saved | Baseline Cost (USD) | With Tool Search (USD) | Net Session Savings | | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | 1 turn | 131,430 | 110,282 | 21,148 tokens | $0.3286 | $0.2757 | $0.0529 (16.09%) | | 5 turns | 657,150 | 551,410 | 105,740 tokens | $1.6429 | $1.3785 | $0.2644 (16.09%) | | 10 turns | 1,314,300 | 1,102,820 | 211,480 tokens | $3.2858 | $2.7571 | $0.5287 (16.09%) | | 20 turns | 2,628,600 | 2,205,640 | 422,960 tokens | $6.5715 | $5.5141 | $1.0574 (16.09%) | | 30 turns | 3,942,900 | 3,308,460 | 634,440 tokens | $9.8573 | $8.2712 | $1.5861 (16.09%) | | 50 turns | 6,571,500 | 5,514,100 | 1,057,400 tokens | $16.4288 | $13.7853 | $2.6435 / session (16.09%) | | 75 turns | 9,857,250 | 8,271,150 | 1,586,100 tokens | $24.6431 | $20.6779 | $3.9653 / session (16.09%) | | 100 turns | 13,143,000 | 11,028,200 | 2,114,800 tokens | $32.8575 | $27.5705 | $5.2870 / session (16.09%) |

4. Academic Research & Local Reproducibility

Explore our full mathematical formulations, proofs, and raw empirical artifacts:

📄 Research Thesis Paper: Mitigating Tool Context Bloat in Multi-Agent LLM Runtimes
Formal proofs on quadratic token accumulation $\mathcal{O}(T^2)$, attention dilution, and the "Tool Bloat Tax".
📐 Benchmark Methodology & IR Standards Specification
Comprehensive evaluation harness specification adhering to TREC, BEIR, and Berkeley Function-Calling (BFCL) standards.
📊 Raw JSON Evaluation Artifacts
Unprocessed empirical run data containing full tool complexity metrics, confidence intervals, and latency distributions.

# Reproduce the full benchmark suite locally (~500ms execution)
npm run bench

Configuration Reference (v1.0.0)

| Option | Type | Default | Description | |---|---|---|---| | alwaysLoad | string[] | [] | Array of tool IDs exempt from deferral (full descriptions always loaded in prompt). | | maxResults | number | 5 | Maximum number of ranked tool matches returned per query. | | mode | 'hybrid' \| 'keyword' | 'hybrid' | Search mode: 'hybrid' (BM25 + ONNX vectors) or 'keyword' (BM25 only). | | resetTools | string[] | ['compress'] | Tool names that trigger authorization reset upon execution (e.g. context compression). | | timeout | number | 60000 | Global MCP server prewarm timeout in ms. Servers exceeding timeout fail-open cleanly. | | mcp.servers | Record<string, McpServerConfig> | {} | MCP server definitions adhering to the OpenCode v2 mcp.servers schema. |

MCP Server Configuration (`mcp.servers`)

Note on Upgrading from v0.2.x: v1.0.0 enforces the OpenCode v2 mcp.servers dictionary wrapper. Legacy bare maps (mcp: { "<server>": {...} }) are rejected.

{
  "plugin": [
    [
      "@openstellar/tool-search@latest",
      {
        "maxResults": 5,
        "timeout": 30000,
        "mcp": {
          "servers": {
            "codebase-memory": {
              "type": "local",
              "command": ["npx", "-y", "codebase-memory-mcp@latest"]
            },
            "remote-docs": {
              "type": "remote",
              "url": "https://mcp.example.com/sse",
              "timeout": 15000
            }
          }
        }
      }
    ]
  ]
}

Architecture & MCP Prewarming

OpenCode Startup
       │
       ▼
 ┌─────────────────────────────────────────────────────────────┐
 │ McpWiring: Parallel Warmup & Connection                     │
 │ ├── server 1 (local stdio)  ──► Settled                     │
 │ ├── server 2 (remote sse)   ──► Settled                     │
 │ └── server 3 (slow/failed)  ──► Status-Only Placeholder     │
 └─────────────────────────────┬───────────────────────────────┘
                               │ (All servers settled or deadlined)
                               ▼
 ┌─────────────────────────────────────────────────────────────┐
 │ Plugin Hook Export: tool.definition                         │
 │ ├── Truncates descriptions to first sentence + [deferred]   │
 │ ├── Inlines JSON parameter schemas                          │
 │ └── Caches full metadata in ToolVault                       │
 └─────────────────────────────┬───────────────────────────────┘
                               │
                               ▼
 ┌─────────────────────────────────────────────────────────────┐
 │ Agent Runtime Execution                                     │
 │ ├── tool_search / tool_search_regex ──► Authorizes Tool     │
 │ └── tool.execute.* ───────────────────► Invokes MCP Bridge  │
 └─────────────────────────────────────────────────────────────┘

Prewarm Before Return: The plugin awaits enabled MCP servers before returning hooks, guaranteeing tools exist in OpenCode's initial immutable session snapshot.
Fail-Open Isolation: If an MCP server crashes or exceeds its timeout, it is marked with a status placeholder tool, allowing all other servers and tools to operate normally without hanging OpenCode.
Atomic Authorization Cache: Authorizations survive CLI restarts via atomic JSON cache with automatic 30-day TTL expiration.

Troubleshooting

| Symptom | Root Cause | Solution | |---|---|---| | Slow startup when opening OpenCode | An enabled MCP server is slow to start. | The plugin awaits servers up to timeout (default 60s). Lower timeout or set a per-server timeout in mcp.servers.<name>.timeout. | | [Tool Search Required] error | The LLM attempted to call a [deferred] tool without searching first. | Call tool_search({ query: "..." }) or tool_search_regex({ pattern: "^name$" }) first. | | MCP server status placeholder shown | Server failed or timed out during prewarm. | Check server command/URL and stderr logs. Once fixed, restart OpenCode. | | Legacy config warning | Using old mcp: { "<server>": {...} } format. | Wrap server definitions in mcp: { servers: { ... } }. | | Need connection/failure details | Server status goes to the plugin's file log (never the terminal — stdout output clobbers the TUI). | Inspect ~/.local/share/opencode/log/tool-search.log. The log self-rotates to tool-search.log.old at 5 MB (max ~10 MB on disk), so it never needs manual cleanup. |

Development & Verification

# Install dependencies
npm install

# Run Vitest test suite (125 tests across 6 suites)
npm test

# Typecheck
npm run typecheck

# Build bundle and roll types
npm run build

# Run isolated npm pack and load smoke test
npm run smoke:plugin