vlmkit

Deterministic verification toolkit for frontend work: markup gates (broken-page scan, copy fidelity, responsive boundaries, keyboard operability, scripted flows), visual regression over time, design audits, and repair tools. Built for humans and coding agents alike — every failing gate prints a machine-parsable fix list. Everything is key-free unless marked [key].

Requires Node 24+.

Quickstart — visual analysis with zero context

npm install -D @mizchi/vlmkit
npx playwright install chromium          # once

# 1. find concrete visual defects on the page you're working on
#    (no reference, no config, no API key — 3 viewports at once)
npx vlmkit check integrity http://localhost:3000/
#    -> verdict: DEFECTS … [page-overflow-x] @768: … main > p (right edge 924px)
#                          [text-collision] @1280: "Total: €1,240" overlaps "Refunds: €80"
#    fix what it names, re-run until: verdict: CLEAN

#    long-polling page: choose an earlier navigation milestone
npx vlmkit check integrity http://localhost:3000/ --wait-until domcontentloaded

# 2. track what your edits change visually (1st run = baseline, then diffs)
npx vlmkit snapshot http://localhost:3000/ --output .vlmkit/snapshots
#    -> desktop 13.11% … mobile 6.57% + diff heatmaps next to the report
#    intended? npx vlmkit snapshot approve --output .vlmkit/snapshots

From there: check copy --manifest (exact copy), check breakpoints --sweep (responsive boundaries), check interactions (keyboard operability), verify markup --target design.png (match a design) — the full task-routing table, done-condition recipes, and agent integration (MCP server + markup-assist skill) live in docs/markup-assist.md.

Demos

mizchi.github.io/vlmkit/demos/ — every feature on a page with a known defect: the command, its whole output, and screenshots from the same run. Most pages are this repository's own test fixtures, so each command works from a clone after pnpm build.

| | Demo | Command | |---|---|---| | | A pricing page that breaks on smaller screens | check integrity | | | Copy that drifted from the spec | check copy --manifest | | | A card grid that needs 826px, switched on at 768px | check responsive | | | A Save button that does nothing on Enter | check interactions | | | A toolbar whose Tab order runs backwards | check a11y focus | | | A checkout button an agent would miss | check grounding --mark | | | Pale text that fails WCAG contrast | check a11y contrast | | | An app with no DOM to read | scan a11y → check a11y tree | | | Headings that sit closer to the section above | check composition | | | Links you can only tell apart by colour | check color | | | Spacing and radii off the design scale | check tokens | | | A header that grew 48px and pushed everything down | diff html | | | Buttons that overflow when the copy gets longer | stress i18n | | | Letting a vision model zoom into the original | @mizchi/vlmkit-ai/zoom.ts |

Six whole sites built by agents, each beside the log of how it was judged, are at /sites/.

Setup

npm install -D @mizchi/vlmkit     # or -g
npx playwright install chromium   # once

vlmkit declares playwright as a peer and @playwright/test as an optional peer, reusing the versions already selected by the project instead of installing a second browser build.

• Coding agents (MCP) — .mcp.json: { "mcpServers": { "vlmkit": { "command": "npx", "args": ["-y", "@mizchi/vlmkit", "mcp"] } } }
• Agent skill — install the automatic vlmkit router once with APM or the open skills CLI (commands below), then ask for the result in natural language.
• API keys — only for [key] features (heal markup, check copy --vlm, fix-loop): OPENROUTER_API_KEY / GEMINI_API_KEY / ANTHROPIC_API_KEY.
• Snapshot / CI config — vlmkit.config.json (routes, thresholds) via vlmkit workflow init.

Full setup detail: docs/configuration.md.

Install agent skills

Recommended: install the vlmkit skill package once. You do not choose a specialized skill; the agent classifies each frontend request, loads the matching bundled workflow, runs its deterministic gates, fixes failures, and reruns to green.

# Install or update APM (macOS / Linux)
curl -sSL https://aka.ms/apm-unix | sh

# Install vlmkit with APM
apm install mizchi/vlmkit

# Or install with the skills CLI (APM is not required)
npx skills add mizchi/vlmkit

Inside Claude Code the same package installs as a plugin — no shell, and /plugin marketplace update picks up new workflows:

/plugin marketplace add mizchi/vlmkit
/plugin install vlmkit@vlmkit

The APM bootstrap above is the official recommended installer and resolves the latest binary for the current platform. It avoids depending on a stale package-manager formula.

Then ask naturally: “implement this mock,” “check this page's responsive and keyboard behavior,” or “turn this story into stable Playwright tests.” There is no skill-selection step in the normal workflow. On first use, the agent detects the repository's package manager, reuses or adds @mizchi/vlmkit locally, and installs Chromium only if a selected gate reports it missing.

All three installers expose one visible vlmkit skill. The 13 specialized workflows are internal resources bundled under that entry, so the agent selects them without adding 13 separate skills or copying the vlmkit source repository.

See the agent skill catalog for all 13 specialized skills, grouped by general verification, UI creation, test generation, comparison and monitoring, and evaluation and hardening. Explanation and figures (animations, D2 diagrams and slides) live in mizchi/explainer.

When to use what

vlmkit --help prints this same map. Everything is key-free unless marked [key].

| You want to… | Reach for | |---|---| | Check the page you just wrote/edited, no reference | check integrity (broken-page scan, 3 viewports) · check copy --manifest (copy present, visibly, verbatim) · check layout --contract · scan scroll / scan handlers | | Verify behavior, not pixels | check breakpoints --sweep · check responsive (every width, shrunk to breakpoint + declaration) · check interactions · check scroll / check animation · verify flow --flow | | Drive the page from a screenshot (computer use) | check grounding — action map in screenshot px, which element a click at each point actually reaches, --mark for a numbered overlay | | Match a target design | verify markup --target (done verdict + fix list) · build page / build component · scan mock (normalize @2x exports) | | Track changes over time (VRT) | snapshot (baseline → per-viewport diff → snapshot approve) · watch · diff-pr · baseline | | Compare two versions | diff html\|png\|elements · migration compare · check equivalence --region | | Audit design quality | check tokens\|theme\|palette · check design (is the page consistent with itself — one button style or six) · check a11y contrast\|touch\|focus · check perf · check drift · stress i18n\|media | | Vet an image asset before it enters a slot | check asset --slot --expect-transparent --against-bg --page-palette | | Repair | heal selector (dead selector) · heal markup [key] (LLM auto-fix from a kickback) | | Run gates over a whole site | batch --gate "check integrity" "routes/**/*.html" (parallel, --shard i/n, per-job timing) · gates run (same, from one reviewed config) | | Audit what has been silenced | gates suppressions — every suppression with reason, owner, expiry; expired ones stop applying | | Wire into agents / pipelines | mcp (gate tools over stdio) · contract · workflow · markup-loop · api · bench · skill |

Task-routing recipes and done-condition sets: docs/markup-assist.md.

With a key: TypeSafe Jev

Four tools ask TypeSafe Jev typed questions — a box, a yes/no (noul), a choice — with TYPESAFEAI_API_KEY. The model says what is there and where; the values come from the pixels and the page. It is never asked to name a measurement.

Jev is opt-in. Where a command works without it, --jev turns it on (diff png --jev, inspect motion --jev). inspect locate, inspect claims and a describe click have no other engine, so they need the key and nothing else.

| Feature | Command | Jev answers | vlmkit measures | Measured | |---|---|---|---|---| | Classify each diff region | diff png --jev | A choice per region: text, colour, position or presence | The regions themselves, from the pixel diff | Text 14/14, colour 18/18, presence 17/17; 1–6px shifts 11/16, every miss marked uncertain | | Find an element from a description | inspect locate, click with describe | Is it shown (noul), a box, then a box again on a magnified crop | The crop, cut from the original pixels; the point, in click coordinates | 94% of targets hit; 0 of 85 decoys accepted | | Check statements about a screen | inspect claims --forbid/--expect | One noul per statement, all in one request | Nothing: it advises and never gates | 54 of 55 planted problems caught; 0 false alarms on 43 clean pages | | Is an animation visibly broken? | inspect motion --jev | Per frame: is each target shown, its box, a noul per forbidden statement | Frames on the held clock; jumps, snap-backs and easing fitted from the boxes; with --selector, geometry from the page | 7/7 broken caught, 7/7 intact passed (deterministic gates: 3/7); ~$0.002 a page. Without --jev, from --selector alone and no key: 6/7 and 7/7 (text left on text needs Jev) |

Asked to name an easing, Jev was at chance (7 of 21); fitted from its boxes, 21 of 21. Reports: region kinds, locate, claims, easing, motion.

The workspace

flowchart BT
  subgraph L0["layer 0"]
    p7["judge"]
  end
  subgraph L1["layer 1"]
    p4["core"]
  end
  subgraph L2["layer 2"]
    p1["ai"]
    p2["animation-eval"]
    p3["capture"]
  end
  subgraph L3["layer 3"]
    p5["generate"]
    p6["heal"]
    p8["markup"]
    p10["plan"]
  end
  subgraph L4["layer 4"]
    p9["mcp"]
  end
  subgraph L5["layer 5"]
    p0["vlmkit (cli)"]
  end
  p0 --> p1
  p0 --> p2
  p0 --> p3
  p0 --> p4
  p0 --> p5
  p0 --> p6
  p0 --> p7
  p0 --> p8
  p0 --> p9
  p0 --> p10
  p1 --> p4
  p1 --> p7
  p2 --> p4
  p2 --> p7
  p3 --> p4
  p4 --> p7
  p5 --> p1
  p6 --> p1
  p6 --> p3
  p6 --> p4
  p8 --> p1
  p8 --> p2
  p8 --> p3
  p8 --> p4
  p8 --> p7
  p9 --> p4
  p9 --> p8
  p10 --> p1

Eleven packages, layer by layer from judge (no dependencies) to the CLI; an arrow points from a package to one it depends on. Generated from the manifests by scripts/workspace-map.mjs, and a test fails when this block drifts from them.

Examples

| | Example | What it shows | |---|---|---| | | Feature demos | One page per feature: a page with a known defect, the command that finds it, its output and screenshots | | | Demo sites | Six sites built by agents with vlmkit's gates, each published beside the log of how it was judged | | | Component-focused VRT | check story on one component instead of a whole page, with a gallery to copy | | | Markup loop project | A minimal standalone project that reproduces the drop-in markup loop | | | Markup VRT eval | A dogfood harness that measures the Playwright workflow while a new screen is built | | | Gate plugin | A project that adds two gates of its own through vlmkit.config.json | | | Solitaire | Klondike in plain HTML/CSS/JS, the drag-and-drop and animation dogfood target | | | Intro page | The dependency-free landing page, verified with vlmkit's own loop |

Documentation

| Doc | Contents | |---|---| | docs/introduce.md | What is vlmkit? — narrative introduction assuming zero context | | docs/markup-assist.md | Task-routing guide: which gate for which job, done-condition recipes, anti-gaming rules | | docs/cli-reference.md | Complete command reference: all groups, examples, workflow/API/HTTP, architecture, project structure | | docs/configuration.md | Install, MCP/skill setup, environment variables, snapshot/CI config, agent skills catalog | | packages/vlmkit-mcp/README.md | MCP tool table | | docs/knowledge.md · docs/reports/ | Accumulated evaluation findings and dated experiment reports | | CHANGELOG.md | Release notes, including the 0.4.x → current command mapping |

License

MIT