A local-first workbench for developing, inspecting, and benchmarking MCP servers against local (LM Studio, Ollama) or remote (OpenRouter) models — one shared workspace with a Web UI for you and a CLI + MCP interface for your coding agent.

Building a good MCP server is empirical work, and you rarely do it alone anymore: a coding agent can run every benchmark and read every trace, but the numbers only get good when the developer stays in the loop — looking at the one run that failed, tightening a tool description or a rubric, deciding what to try next, and re-running. mcpscope is built for that partnership. It is not a benchmark harness an agent runs unattended, and not a GUI a developer drives by hand: every capability is exposed through a web UI shaped for a person and through a CLI and an MCP interface shaped for an agent, all over one shared data model. The human can see exactly what the agent did, the agent can inspect exactly what the human did, and a type-tagged ID system lets both point at the same session, turn, or tool call.

Why that loop matters: an LLM is only as good as the tools it is given — a lesson I learned building my own MCP servers for data analytics and statistics. My first versions did the obvious thing: wrap the existing API and hand the raw data back to the model. That turned out to be the worst possible design — token-hungry, imprecise, and dependent on a large model with a large context window just to produce something useful. Results only became good when the server did the work itself — the calculations, aggregation, and filtering — and returned answers rather than data. That experience is the premise mcpscope is built on: a good MCP server is not an API wrapper. It is a user interface built for an LLM, designed to solve a specific job efficiently even on a small local model. And it is an application like any other, so it starts from clear use cases and quality criteria, and most of ordinary software engineering applies when you design and build it.

The part that does not carry over is testing. You cannot pin end-to-end behavior down with deterministic assertions, because the model is non-deterministic and both its inputs and outputs are mostly natural language. So, like the wider evaluation ecosystem, mcpscope measures quality statistically — run a set of prompts many times against a chosen model and MCP server, then read back per-tool reliability and token cost, with optional scoring of answer quality by a separate judge model. What mcpscope adds is keeping that measurement inside the iteration loop rather than at the end of it: every run stays inspectable down to each reasoning step, tool call, and token of context, so "the score dropped" immediately becomes "look at what the model did here."

You run a prompt, or a repeatable benchmark, against a local (LM Studio, Ollama) or remote (OpenRouter) model, watch every reasoning step, tool call, and token of context, then change one thing (a tool description, a parameter, an output payload) and run it again. Everything stays on your machine.

Get started

Prerequisite (2 minutes): a running LLM backend — local (LM Studio, Ollama) or remote (OpenRouter). For LM Studio: open the Developer tab → Start server, load a model, and note its model id — the server URL is http://localhost:1234/v1.

Desktop app — the easiest way to try mcpscope

Download the installer for your OS from the Releases page (macOS .dmg, Windows .exe, Linux AppImage/.deb/.rpm). Everything is bundled — launch it and the workbench opens; data lives in ~/.mcpscope. While it runs it serves the same backend as mcpscope serve at http://localhost:3066 (MCP interface at /mcp), so coding agents and other MCP clients can connect to it directly — set the BACKEND_HOST / BACKEND_PORT environment variables before launching to change the address (shown in the app under Configuration → Server). The builds are unsigned for now, so macOS Gatekeeper / Windows SmartScreen warn on first run.

npm — for developing an MCP server (adds the CLI)

If you're building an MCP server you'll want the CLI (and the MCP interface for your coding agent). Requires Node.js 24+:

npm install -g mcpscope
mcpscope serve

Or run it without installing: npx mcpscope serve.

mcpscope serve starts mcpscope at http://localhost:3066 and opens it in your browser. Data is stored in ~/.mcpscope; stop with Ctrl-C. Flags: --port <n>, --host <host>, --data-dir <path>, --no-open.

First steps (either install)

1. In the Web UI, open Configuration and add an LM connection (the base URL above) and a model config (the model id you loaded), then set it as the default model. No MCP profile is needed to start — mcpscope ships zero-setup companion MCP servers. Prefer editing a file? The same setup as JSON: CONFIG.md.
2. Create a session, select a companion server (e.g. Open-Meteo Weather), send a prompt, and inspect the full trace: setup, tool definitions, reasoning, tool calls and results, and a color-coded context breakdown per turn.
3. Add an MCP server profile when you are ready to point mcpscope at your own server.
4. Define a benchmark and run it to test an MCP server repeatably across models — then hand the same loop to your coding agent.

The full walkthrough is in TUTORIAL.md; the worked example with real numbers is EXAMPLE.md.

For coding agents

mcpscope speaks MCP itself — connect your agent and it can drive the whole loop on the same sessions you see in the UI:

claude mcp add --transport http mcpscope http://localhost:3066/mcp

{ "mcpServers": { "mcpscope": { "type": "http", "url": "http://localhost:3066/mcp" } } }

Then a prompt like this is enough to put the agent to work:

mcpscope is running at localhost:3066 (MCP at /mcp; the mcpscope CLI is the identical surface). Use its mcpscope_* tools to benchmark my MCP server: start with mcpscope_list_mcp_profiles and mcpscope_list_model_configs, create sessions and send prompts with wait: true so you never poll, and follow the loop in EXAMPLE.md.

Every operation is both a CLI command and an MCP tool (test-enforced parity), results are snake_case JSON, and create/send take wait so agents get terminal results in one call. Details: MCP.md.

Other ways to run

• Docker: a released image is published to GHCR. See TUTORIAL.md for the step-by-step path and RELEASING.md for image tags.
• From source: for working on mcpscope itself, see DEVELOPMENT.md.

What you can do

• Inspect sessions: watch how a model reads tool definitions, reasons, calls tools, and consumes the context window, with auditable token attribution per part.
• Benchmark MCP servers: a reusable suite of prompts run N× against a chosen model and MCP server, producing a per-tool error/usage scorecard and per-case reliability (pass@k / pass^k).
• LLM-evaluate answer quality: a separate judge model scores each run against a per-case rubric (see BENCHMARK.md).
• Drive it from the shell or as MCP tools: every operation is both a mcpscope <cmd> CLI command and a mcpscope_<cmd> MCP tool, so a coding agent can run the whole loop — on the same sessions you see in the UI.

How it compares

Plenty of good tools touch parts of this space; the difference is which part of the loop they serve, and who they keep in it.

• MCP Inspector pokes an MCP server at the protocol level — list tools, call them by hand. There is no model in the loop; mcpscope tests what a model actually does with your server.
• mcpsnoop captures real client↔server MCP traffic on the wire. mcpscope runs the model itself and attributes every token of context.
• MCPJam is the closest neighbor: an LLM playground with evals and traces. mcpscope differs on per-part token attribution, pass@k / pass^k reliability statistics, and the human+agent parity over one shared store.
• promptfoo, mcp-eval, and DeepEval are config- or code-driven eval harnesses, at their best as unattended regression gates in CI. mcpscope is the interactive workbench for the iteration loop before that — and exposes the same operations over CLI/MCP so an agent can drive it too.
• MCPBench, MCP-Bench, and MCPMark benchmark models against fixed sets of servers. mcpscope benchmarks your server against the models you choose.
• LM Studio and Open WebUI are chat interfaces with tool support — for the human only, with aggregate token counts at best. LM Studio is one of mcpscope's supported backends, not a competitor.

If one of these fits your workflow better, use it — several are excellent. mcpscope is for the loop where you and your coding agent iterate on your own MCP server together.

Documentation

Getting started

• TUTORIAL.md - install, configure, run a session, and benchmark an MCP server
• EXAMPLE.md - the worked example: inspect → benchmark → change one thing → watch the metric move
• CONFIG.md - the mcpscope.config.json reference, including fully headless setup
• COMPANIONS.md - bundled zero-config companion MCP servers you can select without any setup
• BENCHMARK.md - benchmark suite/case/run model, deterministic metrics, and LLM rubric evaluation

Interfaces

• MCP.md - MCP interface: transport, tool surface, and structured results
• CLI.md - CLI commands, flags, output format, and exit codes

Internals & contributing

• DEVELOPMENT.md - run from source, build, dev helpers, and repository notes
• AGENTS.md - guide for AI coding agents working on mcpscope itself: project shape, parity principle, working style, validation
• ARCHITECTURE.md - system design, persistence, streaming, replay, and API surface
• PROVIDERS.md - provider internals (LM Studio, Ollama, OpenRouter): reasoning tokens, token counting, context windows, model loading/unloading (incl. auto-swap)
• DATA-MODEL.md - canonical runtime tree, part taxonomy, and IDs
• DATABASE-SCHEMA.md - SQLite tables, foreign keys, and ER diagram
• DESIGN-SYSTEM.md - frontend design system: the tokens, primitives, and patterns that keep the GUI consistent
• design-assets/ - master logo SVGs (logo, mark, wordmark, favicon); see its README
• TESTING.md - test strategy, replay, and how to add regressions
• RELEASING.md - tag-driven release workflow: npm publish, GHCR image, and desktop installers

Internal/contributor docs live under docs/; user-facing guides stay at the repository root.