@inspectr/mcplab-mcp-server
v1.13.1
Published
MCP server that exposes MCPLab evaluation tools — query runs, results, and traces via the Model Context Protocol
Maintainers
Readme
MCPLab MCP Server
MCP server that exposes MCPLab authoring and execution helpers for other LLM clients.
Focus
- Generate MCPLab YAML entries for
servers,agents, and especiallyscenarios - Validate configs via
@inspectr/mcplab-core(loadConfig,selectScenarios) - Run evaluations and inspect run artifacts
- Provide scenario-authoring prompts for MCP-capable clients
Run (Streamable HTTP)
npm run build -w @inspectr/mcplab-mcp-server
node packages/mcp-server/dist/index.jsDefaults:
- Host:
127.0.0.1 - Port:
3011 - MCP endpoint:
/mcp
Override with MCP_HOST, MCP_PORT, and MCP_PATH.
Minimal Manual-Test MCP Servers (Dev Only)
These tiny servers are useful for manual testing in MCP Lab (connectivity, auth, OAuth Debugger). They live outside the published package code in:
/Users/tim.haselaars/Sites/mcp-evaluation/dev/mcp-test-servers
1) Public (no auth)
npm run mcp:test:publicDefaults:
- Base URL:
http://127.0.0.1:3111 - MCP endpoint:
/mcp
2) Bearer-protected
MCP_TEST_BEARER_TOKEN=demo-bearer-token npm run mcp:test:bearerDefaults:
- Base URL:
http://127.0.0.1:3112 - MCP endpoint:
/mcp - Protected probe endpoint:
/probe
3) OAuth mock (for OAuth Debugger)
npm run mcp:test:oauth-mockDefaults:
- Base URL:
http://127.0.0.1:3113 - MCP endpoint:
/mcp(bearer-protected after token exchange) - OAuth endpoints:
/.well-known/oauth-protected-resource/.well-known/oauth-authorization-server/authorize/token/register(DCR)/client-metadata.json(CIMD)/probe(protected resource probe)
Default pre-registered client:
client_id:mcplab-debuggerclient_secret:mcplab-debugger-secretredirect_url:http://localhost:6274/oauth/scope:openid profile mcp
Useful Tools
mcplab_create_test_case— persist a reviewed Test Case YAML file undermcplab/test-cases/mcplab_create_evaluation_config— persist a normalized evaluation YAML file undermcplab/evals/, optionally in a nested foldermcplab_generate_scenario_entrymcplab_validate_configmcplab_list_librarymcplab_get_library_itemmcplab_run_evalmcplab_results_searchmcplab_results_contextmcplab_results_indexmcplab_read_run_artifact
Runtime server selection
Both mcplab_run_eval (immediate execution) and mcplab_queue_run (app queue) accept servers to use MCP servers together, or server_sets to compare sets in one evaluation result:
{
"config_path": "eval.yaml",
"scenario_id": "basic-test",
"runs_per_scenario": 1,
"server_sets": [
{ "id": "baseline" },
{ "id": "candidate", "servers": ["server-v2"] }
]
}Omit servers inside a set to use scenario defaults. For an override without comparison, use "servers": ["server-v2"] instead of server_sets. These two options are mutually exclusive; server sets also cannot be combined with per-scenario overrides.
