getclaros-browser
v0.1.4
Published
Claros guard for browser agents: checks consequential clicks and submits against your team's rules (each with its owner) before they reach the app
Readme
Claros guard for browser agents
Checks a browser agent's consequential clicks and submits against the company's expert-approved Work Maps before they reach the app. Works on any web app: nothing in it knows an app's names, selectors or URLs.
| Adapter | One line |
|---|---|
| Playwright (Node) | import { guard } from "getclaros-browser"; await guard(page); then a stopped page.click(…) throws ClarosStop |
| browser-use (Python, packages/browser-py, browser-use 0.9.2–0.13.11) | from claros_browser import guard_tools; guard_tools(tools) then Agent(…, tools=tools) |
| Playwright MCP / browser MCP servers | not built; see "MCP-level guard" below |
Both read CLAROS_URL and CLAROS_KEY (an agent-scope clr_ key). Options: url, key, timeout (5 s),
failOpen / fail_open (default off: when Claros can't be reached, the action is stopped), askWait / ask_wait
(seconds an ask waits for its person's own answer; default 25, 0: don't wait; past it the agent carries on and the same action goes through once they let it), actor, workflowId,
onNudge / on_nudge.
Install: npm install -D getclaros-browser (Playwright) or pip install claros-browser (browser-use), or let npx getclaros init do it.
What happens
src/page.jsis injected into every frame (PlaywrightaddInitScript; browser-usePage.addScriptToEvaluateOnNewDocument). A capture-phase listener onwindowsees every click, submit and Enter before the app's own handlers.- What counts as an action. Held:
- a submit of a POST form (any button name), including Enter in a field (implicit submit);
- a click (or Enter / Space on a custom
role=button) on a button, link or menu item whose accessible name (aria-labelledby,aria-label, text,value,title) says commit: submit, save, send, pay, post, approve, reject, delete, remove, deploy, merge, confirm, publish, refund, transfer, update, create … in English, German, French, Spanish, Portuguese, Italian and Russian (words and stems). "Cancel" / "Close" alone dismiss; with an object ("Cancel order", "Close ticket") they commit. A name starting with next / back / search / preview / edit / open / sign in … never commits. Inside a dialog, OK / Yes / Continue commit. A button inside a form's table row ("Remove line") edits the form and isn't held; - anything Claros's tool kinds for the app (
GET /v1/tools/kinds?app=<origin>, expert or LLM classified, keyed by the button's name) call a write; a name they call a read is never held. Reads, navigation and typing pass through untouched. The app's handlers and the default action never run for a held event.
- Reading the record at that moment, from the DOM:
- scope: the button's form if it holds fields, else the nearest ancestor that does (a field-less confirm dialog or a bottom action bar → the screen around it), else the page;
- labels:
aria-labelledby,aria-label,<label>(for / wrapping),placeholder,title, the nearest text before the control,name; - values: inputs, textareas, selects (the option shown), checkboxes and switches (1 / 0), radio groups (the checked
option's label, under the group's legend), contenteditable,
role=textbox|combobox|spinbutton|listbox,<output>; - line items: tables and ARIA grids become rows of {column header: cell}; repeated labels outside a table become
rows by order;
<dl>and two-column header tables become label → value (a wizard's review step); - open shadow roots are walked; unrendered fields are skipped outside forms (a form submits its hidden steps).
- The harness takes the held action and asks Claros:
POST /v1/check/page {app: origin, action: the button's name, state: {label: value}, rows: [{label: value}], url, title}. The server maps labels to canonical vars through the maps' aliases (any language) and routes like/v1/check. If the DOM gave no fields at all (a canvas app, a cross-origin iframe), a screenshot goes toPOST /v1/check/screeninstead: the people product's screen reader (redaction + OCR + one vision read → ScreenState) reads it on the server. - Allowed → the action is replayed (
el.click()/form.requestSubmit(submitter)/ the key) and reaches the app once. Ask (a rule that says ask first) → the question goes to the person the agent works for (Let it / Stop it / Ask in their notch). The held action waits up toaskWaitseconds (default 25; 0 = don't) onGET /v1/check/{id}/decision?payload_sha256=…and is replayed only if they let this exact action through (the hash Claros took of it); Stop it is a stop with their reason. No answer by then, or an ask that is with the rule's owner (an "Only I can approve this" rule: no wait): the action is dropped andClarosStopcarries Claros's message (who was asked; carry on; the same action goes through when it's tried again once they let it). The Playwright call blocks while it waits, so give a Playwright Test that can hit an ask a longertest.setTimeout. Stopped (stop, or an ask not approved) → dropped; the Playwright call that caused it throwsClarosStop(rule,quote,expert,whoToAsk,checkId,result;messageis the same text the MCP guard returns); in browser-use the action returnsActionResult(error=…)with that text, so the agent sees the rule, the expert's words and who to ask. Claros unreachable, erroring or slower thantimeout→ stopped with "Claros couldn't check …". Nudged (onlynudgerules broken, e.g. taste) → replayed like an allow; the nudge ("Claros nudge: …", the rule, the expert's words, the suggested fix) goes to Playwright'sonNudge(n)option,guard.nudgesand aconsole.warn; in browser-use it is appended to the action result'sextracted_content(kept in the agent's memory) and logged as a warning, pluson_nudge(n)/Guard.nudges. - Playwright only: a POST navigation that no held action released (
form.submit()from a script fires no submit event) is checked from its form data (field names) and aborted on a stop.
Playwright wraps click, dblclick, tap, press, check, uncheck, setChecked, dispatchEvent, type,
pressSequentially on the page and on every Locator, plus keyboard.press/type/down and mouse.click/dblclick/up:
after the call they wait for the checks it caused and throw a stop. Anything else that triggers a commit
(page.evaluate) is still held and checked; its stop lands in guard.stops and onDecision.
Develop
npm install
npm test # node:test, real Chromium: action detection, label extraction on the three test apps, stop / allow,
# Enter paths, form.submit(), fail closed (unreachable / 500 / timeout), --fail-open, screen fallback,
# tool kinds; a fake Claros
npm run e2e # a fresh local Claros (temp DB, demo maps in fixtures/maps), the three test apps, Playwright, the
# browser-use scripted controller (packages/browser-py/e2e/run.py);
# CLAROS_E2E_DOTENV=<.env with LLM keys> adds the screen reader's vision read and a real browser-use
# Agent run
cd ../browser-py && uv run pytest # the Python client and guard_tools on a real browser-use sessionclaros_browser/page.js is a copy of src/page.js (a pytest fails when they differ): after editing, cp
src/page.js ../browser-py/claros_browser/page.js.
Test apps (fixtures/apps, served by fixtures/server.mjs on separate origins with a request log): an expense report
(plain HTML form, POST navigation, line-item table), a refund request (SPA, custom ARIA components, no <form>,
fetch), a production deploy request (wizard whose steps unmount, review summary, confirm dialog), and a canvas page
for the screen fallback. Maps in fixtures/maps are hand-written demo data.
MCP-level guard (not built)
getclaros guard can already wrap @playwright/mcp (stdio). Its tools aren't in the read-name list, so each
browser_click / browser_type / browser_fill_form call is checked, but the call carries only {element, ref}
(and typed text): the record is in the page, not in the arguments, so routing finds no map and the call is allowed. What it would take, all as mapping data: a fetch whose read tool returns the page's record
(browser_evaluate running page.js's record() for the ref), classified read by the expert for that one script,
and an actions entry that turns browser_click on a commit-named element into an action. Not small; the in-page
hook above covers browser agents that run their own Playwright or browser-use.
Limits
- Detection is by name and form method. A commit button named only by an icon with no accessible name, or an app
that commits on
mousedown/pointerup/ a custom key, isn't held. Tool kinds (by button name) fix a known one. fetch/ XHR writes not caused by a held click (a background autosave) aren't checked.- The record is what the page shows next to the button: the nearest container with fields. A search box beside the button would be taken for the record; values only in hidden state (not on screen, not in the form) aren't seen.
- Screen fallback: the vision read keeps 5 grid rows; without an LLM key the server reads OCR pairs only.
- browser-use: the main frame only (held actions in iframes stay held: fail closed); no network-level check of
form.submit(). A held action is lost if the page navigates before the check finishes (it never reached the app). - One
guard(page)per page; guard new tabs / popups yourself (context.on("page", (p) => guard(p))).
