@keboola/validate-ui

Closed-loop validation for agent-generated Keboola UI. It boots a rendered module in a headless browser, drives it, and returns a machine-readable verdict across four axes so an agent can see and fix its own output before a human looks:

• runtime-health — white-screens, console/page errors, failed requests
• a11y — axe-core violations mapped to WCAG
• visual-brand — baseline diff + the alt-brand fitness function (chrome must re-skin under an alternate brand) + layout overflow, blocking below 768px wide (asserting categorical colors stay fixed is future work)
• brief-conformance — a VLM judge on "did it build what the brief asked, and are the empty/loading/error states present?"

Static gates (tsc, oxlint, brand-audit) prove code compiles and is brand-safe. This proves the UI actually works and looks right.

Install

Public on npm. Outside the monorepo:

pnpm add -D @keboola/validate-ui
pnpm exec playwright install chromium

CLI

# Validate a running URL
validate-ui --url http://localhost:5173 --brief "Users list with empty state"

# Serve a built SPA dir and validate a route
validate-ui --serve apps/boilerplate/dist --route /tokens --json

# Judge the page at a phone size instead of the 1280x800 default
validate-ui --url http://localhost:5173 --viewport 375x812

Exit code is 0 when every axis passes, 1 otherwise.

--viewport takes WIDTHxHEIGHT or one of desktop | mobile | phone. Below 768px wide a horizontal layout overflow is a critical finding rather than a moderate one: the reader has to scroll sideways to finish a sentence, so the screen is broken and the verdict fails. Omitting the flag renders at 1280x800 and is unchanged.

Programmatic

import { validate } from '@keboola/validate-ui';

const verdict = await validate({
  url: 'http://localhost:5173/tokens',
  brief: 'A paginated tokens list with search and an empty state',
});

if (!verdict.pass) {
  for (const axis of verdict.verdicts.filter((a) => !a.pass)) {
    console.error(axis.axis, axis.findings);
  }
}

capture() and capturePage() expose the render/capture primitive; runAxes() and the individual axes are exported for custom pipelines.

brief-conformance VLM backend

The brief-conformance axis judges the screenshot + DOM with a Claude vision model. It picks a backend from the environment, and skips gracefully (a minor finding, no failure) when none is configured:

• Keboola LLM path (preferred for internal/team use, no dedicated API key needed) — set VALIDATE_UI_LLM_BASE_URL to the Keboola-hosted Claude proxy and VALIDATE_UI_LLM_TOKEN to your Keboola token. This points the Anthropic SDK at the proxy (baseURL) and authenticates with the token (x-api-key), the same routing kai-agent uses. The SDK-native ANTHROPIC_BASE_URL + ANTHROPIC_API_KEY pair — the exact vars kai-agent injects to route the proxy — is also accepted; VALIDATE_UI_LLM_TOKEN and KBC_TOKEN are Keboola-convention aliases for the token. Token precedence: VALIDATE_UI_LLM_TOKEN → KBC_TOKEN → ANTHROPIC_API_KEY; base-URL precedence: VALIDATE_UI_LLM_BASE_URL → ANTHROPIC_BASE_URL.
• Raw key path — set ANTHROPIC_API_KEY with no proxy base URL for a direct Anthropic API call.

When both a proxy base URL and a raw key are set, the Keboola LLM path wins. Override the model with VALIDATE_UI_MODEL (default claude-sonnet-4-6).