@geometra/mcp

MCP server for Geometra — interact with running Geometra apps via the geometry protocol over WebSocket. For native Geometra apps there is no browser in the loop. For any existing website, use geometra_connect with pageUrl — the MCP server starts @geometra/proxy for you (bundled dependency) so you do not need a separate terminal or a ws:// URL. You can still pass url: "ws://…" if a proxy is already running, and if you accidentally pass https://… in url, MCP will treat it as pageUrl and auto-start the proxy.

See AGENT_MODEL.md for the MCP mental model, why token usage can be lower than large browser snapshots, and how headed vs headless proxy mode works.

What this does

Connects Claude Code, Codex, or any MCP-compatible AI agent to a WebSocket endpoint that streams frame / patch messages. The agent gets structured { x, y, width, height } geometry and semantic metadata for every UI node — not screenshots, not a vision model.

Playwright + vision:  screenshot → model → guess coordinates → click → repeat
Native Geometra:      WebSocket → JSON geometry (no browser on the agent path)
Geometra proxy:       Chromium → DOM geometry → same WebSocket as native → MCP tools unchanged (often started via `pageUrl`, no manual CLI)

When to choose this

Use Geometra MCP when an LLM needs to explore, interpret, and operate a real UI with compact semantic state instead of repeatedly consuming large browser snapshots. Keep Playwright-style tooling for deterministic scripts, DOM-oriented test automation, and compatibility fallback paths while Geometra closes remaining live-site gaps.

Proxy-backed sessions stay warm by default on disconnect, and MCP now keeps a small warm pool so compatible headed, headless, and stealth workflows do not immediately evict each other.

Tools

| Tool | Description | |---|---| | geometra_connect | Connect with url (ws://…) or pageUrl (https://…) to auto-start geometra-proxy; pass stealth: true for CloakBrowser's patched Chromium; can inline formSchema and/or pageModel, or defer the page model for a faster first connect response | | geometra_prepare_browser | Pre-launch and pre-navigate a reusable proxy/browser for pageUrl without creating an active session; supports the same headed/headless/stealth browser settings as geometra_connect | | geometra_query | Find elements by stable id, role, name, text content, prompt/section/item context, current value, or semantic state such as invalid, required, or busy | | geometra_wait_for | Wait for a semantic condition instead of guessing sleeps (busy, disabled, alerts, values, etc.). Strict parameters — use text plus present: false to wait until a substring disappears (e.g. “Parsing your resume”); there is no textGone field | | geometra_wait_for_resume_parse | Convenience wait until a parsing banner is gone (defaults: text: "Parsing", present implied false). Same engine as geometra_wait_for | | geometra_form_schema | Compact, fill-oriented form schema with stable field ids and collapsed radio/button groups; can auto-connect from pageUrl / url | | geometra_fill_form | Fill a form from valuesById / valuesByLabel in one MCP call; can auto-connect from pageUrl / url for the lowest-token known-form path | | geometra_fill_fields | Fill text/choice/toggle/file fields in one MCP call; text-only batches now use the proxy fast path when step output is omitted, and text/choice/toggle entries can use fieldId from geometra_form_schema without repeating the label | | geometra_run_actions | Execute a batch of high-level actions in one MCP round trip; can auto-connect from pageUrl / url and return a final-only payload for the smallest multi-step responses | | geometra_page_model | Summary-first webpage model: archetypes, stable section ids, counts, top-level sections, and primary actions with nearby section/item context when available | | geometra_find_action | Resolve a repeated button/link by action label plus optional sectionText, promptText, or itemText before clicking | | geometra_expand_section | Expand one form/dialog/list/landmark from geometra_page_model on demand, with paging/filtering for long sections | | geometra_reveal | Scroll until a matching node is visible instead of guessing wheel deltas; auto-scales reveal steps for tall forms when omitted | | geometra_click | Click by coordinates or semantic target, including repeated-action disambiguation with promptText, sectionText, or itemText, optionally waiting for a post-click semantic condition | | geometra_type | Type text into the focused element | | geometra_key | Send special keys (Enter, Tab, Escape, arrows) | | geometra_upload_files | Attach files: labeled field / auto / hidden input / native chooser / synthetic drop (@geometra/proxy only) | | geometra_pick_listbox_option | Pick an option from a custom dropdown/searchable combobox; can open by field label, falls back to keyboard selection, and returns visible option hints on failure (@geometra/proxy only) | | geometra_select_option | Choose an option on a native <select> only; use geometra_pick_listbox_option for custom dropdowns (@geometra/proxy only) | | geometra_set_checked | Set a checkbox or radio by label instead of coordinate clicks (@geometra/proxy only) | | geometra_wheel | Mouse wheel / scroll (@geometra/proxy only) | | geometra_snapshot | Default compact: flat viewport-visible actionable nodes (minified JSON). view=full for nested tree, view=form-required for required fields including offscreen ones | | geometra_layout | Raw computed geometry for every node | | geometra_disconnect | Close the connection |

geometra_wait_for and loading banners

The tool input is strict: unknown keys are rejected (so mistaken names like textGone fail fast instead of being ignored). To wait until copy such as “Parsing…” or “Parsing your resume” is gone, pass a substring that matches the banner in text and set present to false. Example: { "text": "Parsing", "present": false } (adjust the substring per site).

For the common resume-upload case, prefer geometra_wait_for_resume_parse (optional text, default "Parsing"; optional timeoutMs, default 60000) so agents do not have to remember present: false.

Custom dropdowns and searchable comboboxes

Use geometra_pick_listbox_option for React Select / Radix / Headless UI / custom ATS dropdowns. Prefer fieldLabel so MCP opens the right combobox semantically, and pass query when the dropdown is searchable. If click-selection cannot be confirmed, the proxy now falls back to keyboard navigation for editable comboboxes.

When selection fails, the error payload includes a capped visibleOptions list so an agent can retry with a real label instead of blindly guessing. Use geometra_select_option only for native HTML <select> controls.

Long forms and offscreen required fields

For tall application forms, geometra_snapshot({ view: "form-required" }) returns required fields even when they are offscreen, with bounds, visibility, and scrollHint data. This is useful for “what required fields are left?” without paging through a full tree.

When a target is below the fold, geometra_reveal and semantic click actions auto-scale reveal steps when maxSteps / maxRevealSteps are omitted, so long forms do not fail after a hard-coded short scroll budget. If you already have schema ids, geometra_fill_fields can now use fieldId directly for text / choice / toggle entries without repeating fieldLabel / label.

Setup

One-line install:

claude mcp add geometra -- npx -y @geometra/mcp

Uninstall:

claude mcp remove geometra

Or manually add to .mcp.json (project-level) or ~/.claude/settings.json (global):

{
  "mcpServers": {
    "geometra": {
      "command": "npx",
      "args": ["-y", "@geometra/mcp"]
    }
  }
}

To uninstall manually, remove the geometra entry from the config file.

Add to your Claude Desktop MCP config:

{
  "mcpServers": {
    "geometra": {
      "command": "npx",
      "args": ["-y", "@geometra/mcp"]
    }
  }
}

To uninstall, remove the geometra entry from the config file.

Add to your Codex MCP configuration:

{
  "mcpServers": {
    "geometra": {
      "command": "npx",
      "args": ["-y", "@geometra/mcp"]
    }
  }
}

To uninstall, remove the geometra entry from the config file.

Open Settings → MCP → Add new MCP server, or add to .cursor/mcp.json:

{
  "mcpServers": {
    "geometra": {
      "command": "npx",
      "args": ["-y", "@geometra/mcp"]
    }
  }
}

To uninstall, remove the entry from MCP settings.

Add to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "geometra": {
      "command": "npx",
      "args": ["-y", "@geometra/mcp"]
    }
  }
}

To uninstall, remove the entry from the config file.

One-line install:

code --add-mcp '{"name":"geometra","command":"npx","args":["-y","@geometra/mcp"]}'

Or add to .vscode/mcp.json:

{
  "servers": {
    "geometra": {
      "command": "npx",
      "args": ["-y", "@geometra/mcp"]
    }
  }
}

To uninstall, remove the entry from MCP settings or delete the server from the MCP panel.

Any MCP client that supports stdio transport can use Geometra. The server config is:

{
  "command": "npx",
  "args": ["-y", "@geometra/mcp"]
}

To uninstall, remove the server entry from your client's MCP configuration.

From source (this repo)

cd mcp
npm install
npm run build
claude mcp add geometra -- node ./dist/index.js

Usage

Native Geometra server

cd demos/server-client
npm run server  # starts on ws://localhost:3100

Any web app (Geometra proxy)

Preferred path for agents: call geometra_connect({ pageUrl: "https://…" }) and let MCP spawn the proxy for you on an ephemeral local port. The manual CLI below is still useful for debugging or when you want to inspect the proxy directly.

In one terminal, serve or open a page (example uses the repo sample):

cd demos/proxy-mcp-sample
python3 -m http.server 8080

In another terminal (from repo root after npm install / bun install and bun run build):

npx geometra-proxy http://localhost:8080 --port 3200
# Requires Chromium: npx playwright install chromium
# Optional stealth prefetch: npx cloakbrowser install

geometra-proxy opens a visible Chromium window by default. For servers or CI, pass --headless or set GEOMETRA_HEADLESS=1. Pass --stealth, stealth: true on MCP tools, or GEOMETRA_STEALTH=1 to use CloakBrowser's patched Chromium through the same proxy protocol. Optional --slow-mo <ms> slows Playwright actions so they are easier to watch. Headed vs headless usually does not materially change token usage, since token usage is driven by MCP tool output rather than whether Chromium is visible.

Point MCP at ws://127.0.0.1:3200 instead of a native Geometra server. The proxy translates clicks and keyboard messages into Playwright actions and streams updated geometry.

Then in Claude Code (either backend):

> Connect to my Geometra app at ws://localhost:3100 and tell me what's on screen

> Give me the page model first, then expand the main form

> Click the "Submit" button

> Type "[email protected]" into the email input

> Take a snapshot and check if the dialog is visible

Example: agent testing a signup form (native server)

Agent:  geometra_connect({ url: "ws://localhost:3100" })
        → Connected. UI: button "Sign Up", textbox "Email", textbox "Password"

Agent:  geometra_click({ role: "textbox", name: "Email" })
        → Clicked textbox "Email" (n:0) at (250, 220). Email input focused.

Agent:  geometra_type({ text: "[email protected]" })
        → Typed. Email field updated.

Agent:  geometra_click({
          role: "button",
          name: "Sign Up",
          waitFor: { role: "dialog", name: "Success" }
        })
        → Clicked. Success message visible.

No screenshots and no vision model in the loop. JSON in, JSON out.

Example: static HTML via @geometra/proxy

With python3 -m http.server 8080 in demos/proxy-mcp-sample and npx geometra-proxy http://localhost:8080 --port 3200 running:

Agent:  geometra_connect({ url: "ws://127.0.0.1:3200" })
        → Connected. UI includes textbox "Email", button "Save", …

Agent:  geometra_form_schema({})
        → {"forms":[{"formId":"fm:1.0","fields":[{"id":"ff:1.0.0","label":"Email"}, ...]}]}

Agent:  geometra_fill_form({
          formId: "fm:1.0",
          valuesByLabel: { "Email": "[email protected]" },
          failOnInvalid: true
        })
        → {"completed":true,"successCount":1,"errorCount":0,"final":{"invalidCount":0,...}}

Agent:  geometra_query({ role: "button", name: "Save" })
        → Click center to submit the sample form; status text updates in the DOM

How it works

1. The MCP server connects to a WebSocket peer that speaks GEOM v1 (frame with layout + tree, optional patch updates).
2. It receives the computed layout ({ x, y, width, height } for every node) and the UI tree (kind, semantic, props, handlers, children).
3. It builds an accessibility tree from that data — roles, names, focusable state, bounds.
4. geometra_snapshot defaults to a compact flat list of viewport-visible actionable nodes (minified JSON) to reduce LLM tokens; use view: "full" for the complete nested tree or view: "form-required" for offscreen required fields plus scroll hints.
5. geometra_form_schema is the compact form-specific path: stable field ids, required/invalid state, current values, and collapsed choice groups without layout-heavy section detail. It can auto-connect when you pass pageUrl / url.
6. geometra_fill_form turns a compact values object into semantic field operations server-side, so the model does not need to emit one tool call per field. It can also auto-connect, which collapses “open page + fill known values” into a single tool call.
7. geometra_fill_fields is the lower-level field-intent path when you need direct control; text / choice / toggle entries can now resolve fieldId from geometra_form_schema without repeating labels.
8. geometra_page_model is still the right summary-first path for non-form exploration: page archetypes, stable section ids, counts, top-level landmarks/forms/dialogs/lists, and a few primary actions with nearby context.
9. geometra_find_action is the narrow repeated-action resolver when the page has many identical buttons/links and you want one exact target by itemText, sectionText, or promptText.
10. geometra_expand_section fetches richer details only for the section you care about (fields, actions, headings, nested lists, list items, text preview).
11. After interactions, action tools return a semantic delta when possible (dialogs opened/closed, forms appeared/removed, list counts changed, named/focusable nodes added/removed/updated). If nothing meaningful changed, they fall back to a short current-UI overview.
12. After each interaction, the peer sends updated geometry (full frame or patch) — the MCP tools interpret that into compact summaries.

Long Forms

For long application flows, prefer one of these patterns:

1. geometra_fill_form({ pageUrl, valuesByLabel }) when you already know the fields you want to set
2. otherwise geometra_form_schema
3. then geometra_fill_form
4. geometra_snapshot({ view: "form-required" }) when you need the remaining required fields including offscreen ones
5. geometra_reveal for far-below-fold targets such as submit buttons (auto-scales reveal steps when you omit maxSteps)
6. geometra_click({ ..., waitFor: ... }) when one action should also wait for the next semantic state
7. geometra_prepare_browser({ pageUrl, headless, stealth, width, height }) when you can hide browser startup before the real task and want the next proxy-backed flow to attach warm
8. geometra_run_actions when you need mixed navigation + waits + field entry, especially with pageUrl / url for a one-call flow
9. geometra_connect({ pageUrl, returnPageModel: true }) when you want connect + summary-first exploration in one turn
10. geometra_connect({ pageUrl, returnPageModel: true, pageModelMode: "deferred" }) when first-response latency matters more than inlining the page model; follow with geometra_page_model
11. geometra_find_action or geometra_click({ name, itemText / sectionText / promptText, ... }) when the target action repeats across cards/rows
12. geometra_page_model + geometra_expand_section when you are still exploring the page rather than filling it

Typical batch:

{
  "actions": [
    { "type": "click", "role": "textbox", "name": "Full name" },
    { "type": "type", "text": "Taylor Applicant" },
    { "type": "upload_files", "paths": ["/Users/you/resume.pdf"], "fieldLabel": "Resume" },
    { "type": "click", "role": "button", "name": "Submit", "waitFor": { "text": "Application submitted", "timeoutMs": 10000 } }
  ]
}

Single action tools default to short human-readable summaries (detail: "minimal"). Use detail: "terse" for the smallest machine-friendly JSON responses, or detail: "verbose" when you need a fuller current-UI fallback for debugging.

For the smallest long-form responses, prefer:

1. detail: "terse" for compact machine-friendly action responses
2. includeSteps: false when you only need aggregate success/error counts plus the final validation/state payload
3. output: "final" on geometra_run_actions when you only need completion state plus the final semantic signals
4. geometra_prepare_browser before the flow when you can shift browser launch out of the user-visible task window

Typical low-token form fill:

{
  "formId": "fm:1.0",
  "valuesById": {
    "ff:1.0.0": "Taylor Applicant",
    "ff:1.0.1": "[email protected]",
    "ff:1.0.2": "Germany",
    "ff:1.0.3": "No"
  },
  "failOnInvalid": true,
  "includeSteps": false,
  "detail": "minimal"
}

For long single-page forms:

1. Use geometra_expand_section with fieldOffset / actionOffset to page through large forms instead of taking a full snapshot.
2. Add onlyRequiredFields: true or onlyInvalidFields: true when you want the actionable subset.
3. Use contextText in geometra_query / geometra_wait_for to disambiguate repeated Yes / No controls by question text.
4. Use geometra_reveal instead of manual wheel loops when the next target is offscreen.

Typical field fill:

{
  "fields": [
    { "kind": "text", "fieldLabel": "Full name", "value": "Taylor Applicant" },
    { "kind": "text", "fieldLabel": "Email", "value": "[email protected]" },
    { "kind": "choice", "fieldLabel": "Country", "value": "Germany" },
    { "kind": "choice", "fieldLabel": "Will you require sponsorship?", "value": "No" },
    { "kind": "file", "fieldLabel": "Resume", "paths": ["/Users/you/resume.pdf"] }
  ],
  "failOnInvalid": true,
  "detail": "minimal",
  "includeSteps": false
}

With a native Geometra server, layout comes from Textura/Yoga. With @geometra/proxy, layout comes from the browser’s computed DOM geometry; the MCP layer is the same.