@sensoit/mcp-server

An MCP (Model Context Protocol) server for Sensoit that lets you query traces, run evaluations, and manage prompts directly from your IDE or AI assistant.

Features

• Query Traces: Search and analyze runtime events with filters
• Run Evaluations: Trigger and check eval results for prompt versions
• Check Guardrails: Test text against your guardrail policies
• Monitor Costs: Get cost summaries and projections
• Track Incidents: List and review alerts and incidents
• Query Judge Verdicts: Analyze LLM judge decisions

Quick Start

1. Install

npm install -g @sensoit/mcp-server

Or run directly with npx:

npx @sensoit/mcp-server

2. Get Your API Key

1. Go to guard.sensoit.io/settings/keys
2. Create a new API key
3. Copy the key (starts with gf_live_ or gf_test_)

3. Configure Your IDE

Choose your IDE below for setup instructions.

IDE Setup

VS Code

Create or edit .vscode/mcp.json in your workspace:

{
  "mcpServers": {
    "sensoit": {
      "command": "npx",
      "args": ["@sensoit/mcp-server"],
      "env": {
        "SENSOIT_API_KEY": "gf_live_your_api_key_here"
      }
    }
  }
}

Then restart VS Code or reload the window.

Cursor

1. Open Cursor Settings (Cmd/Ctrl + ,)
2. Search for "MCP"
3. Add a new MCP server:

{
  "sensoit": {
    "command": "npx",
    "args": ["@sensoit/mcp-server"],
    "env": {
      "SENSOIT_API_KEY": "gf_live_your_api_key_here"
    }
  }
}

Claude Desktop

Edit your claude_desktop_config.json:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "sensoit": {
      "command": "npx",
      "args": ["@sensoit/mcp-server"],
      "env": {
        "SENSOIT_API_KEY": "gf_live_your_api_key_here"
      }
    }
  }
}

Then restart Claude Desktop.

Windsurf

Add to your Windsurf MCP settings:

{
  "sensoit": {
    "command": "npx",
    "args": ["@sensoit/mcp-server"],
    "env": {
      "SENSOIT_API_KEY": "gf_live_your_api_key_here"
    }
  }
}

Available Tools

Once configured, your AI assistant can use these tools:

Prompt Management

| Tool | Description | |------|-------------| | sensoit_list_prompts | List all prompts with optional search | | sensoit_get_prompt | Get prompt by ID with active version | | sensoit_get_version | Get specific prompt version content |

Traces & Monitoring

| Tool | Description | |------|-------------| | sensoit_search_traces | Search runtime events with filters | | sensoit_get_trace | Get full trace/session detail | | sensoit_get_costs | Get cost summary for date range | | sensoit_list_incidents | List recent incidents/alerts |

Evaluation

| Tool | Description | |------|-------------| | sensoit_run_eval | Trigger an eval run for a prompt version | | sensoit_get_eval_results | Get eval run results |

Safety

| Tool | Description | |------|-------------| | sensoit_check_guardrail | Test text against guardrail policies | | sensoit_get_judge_verdicts | Query judge verdict history |

Example Prompts

Here are some things you can ask your AI assistant:

Debugging & Monitoring

"Show me traces with guardrail violations from today"

"What errors occurred in the last hour?"

"Find traces with latency over 2 seconds"

"Show me the most expensive API calls this week"

Evaluation

"What's the accuracy of my support-bot prompt?"

"Run evals on the latest version of checkout-flow"

"Show me the failed test cases for the last eval run"

Safety Testing

"Check if this text would trigger any guardrails: [paste text]"

"Would this message be blocked by our toxicity filter?"

"Test this user input against our content policies"

Cost Analysis

"How much have we spent on API calls this month?"

"Show me the cost breakdown by prompt"

"What's our projected monthly spend?"

Incident Management

"Are there any open incidents?"

"Show me critical alerts from the last 24 hours"

"What incidents were resolved this week?"

Resources

The server also exposes these MCP resources:

| URI | Description | |-----|-------------| | sensoit://prompts | List of all prompts | | sensoit://prompts/{id} | Specific prompt details | | sensoit://traces/{id} | Specific trace data | | sensoit://incidents/recent | Recent incidents |

Environment Variables

| Variable | Required | Description | |----------|----------|-------------| | SENSOIT_API_KEY | Yes | Your Sensoit API key | | SENSOIT_BASE_URL | No | Override API base URL (default: https://api.sensoit.io) |

Development

To build from source:

# Clone the repo
git clone https://github.com/sensoit/sensoit-product-main.git
cd sensoit-product-main/packages/mcp-server

# Install dependencies
npm install

# Build
npm run build

# Run locally
SENSOIT_API_KEY=your_key node dist/index.js

Troubleshooting

"SENSOIT_API_KEY is required"

Make sure you've set the environment variable in your IDE configuration.

"API Error: Invalid API key"

Check that your API key is valid and hasn't expired. Generate a new one at guard.sensoit.io/settings/keys.

Server not appearing in IDE

1. Make sure you've restarted your IDE after adding the configuration
2. Check that npx is available in your PATH
3. Look for error messages in your IDE's output/logs

Connection timeout

If you're behind a corporate firewall, you may need to configure proxy settings in the environment.

Security

• API keys are passed via environment variables, never hardcoded
• The server only makes requests to api.sensoit.io (or your configured base URL)
• All communication uses HTTPS
• No data is logged or stored by the MCP server

Support

• Documentation: docs.sensoit.io/mcp-server
• Issues: github.com/sensoit/sensoit-product-main/issues
• Discord: discord.gg/sensoit

License

MIT - Sensoit Private Limited