spark-ui-mcp

MCP server for the Apache Spark UI REST API. Lets AI assistants (Claude, etc.) inspect running Spark applications — jobs, stages, SQL executions, executors, environment and storage.

Supports SOCKS5 proxy for reaching Spark UIs inside Kubernetes or behind firewalls.

Install

npm install -g spark-ui-mcp

Usage with Claude Code

Add to your Claude Code MCP settings (~/.claude/settings.json):

{
  "mcpServers": {
    "spark-ui": {
      "command": "spark-ui-mcp",
      "env": {
        "SOCKS5_PROXY": "socks5://localhost:1080"
      }
    }
  }
}

The SOCKS5_PROXY environment variable is optional — only needed when the Spark UI is not directly reachable (e.g. inside a Kubernetes cluster). DNS resolution is delegated to the proxy automatically.

Tools

| Tool | Description | |------|-------------| | spark_list_apps | List Spark applications | | spark_get_jobs | List jobs (optionally filter by status) | | spark_get_stages | List stages (optionally filter by status) | | spark_get_stage | Get stage details with task metrics | | spark_get_sql_queries | List SQL executions | | spark_get_sql_execution | Full SQL execution details: DAG, physical plan, metrics | | spark_get_executors | Executor information | | spark_get_environment | Spark configuration and environment | | spark_get_storage | RDD/storage information |

Every tool takes a sparkUrl parameter — the base URL of the Spark UI (e.g. http://spark-app:4040).

SOCKS5 Proxy

To access Spark UIs inside Kubernetes, set up a SOCKS5 proxy with kubectl:

kubectl port-forward svc/your-proxy 1080:1080

Then set SOCKS5_PROXY=socks5://localhost:1080.

License

MIT