lookout

A project-agnostic visual AI tester. lookout captures what an app actually renders (web pages across form factors and color schemes; iOS simulator and Android emulator screens), judges the evidence against UI best practices and per-project rules through the locally installed Claude Code CLI, verifies acceptance criteria from a ticket, answers free-form fact-check questions, and tracks findings in an adjudicated per-project backlog.

lookout is a pure oracle: it never edits code and never starts services. Agents (or humans) working in the target repo do the fixing; lookout verifies.

Requirements

• bun (the CLI runs under bun so TypeScript configs are first-class)
• Claude Code logged in locally (judging shells out to claude -p)
• Playwright chromium: bunx playwright install chromium
• Optional, for native capture: Xcode command line tools (iOS simulator) and Android platform-tools (adb)

Check everything with:

lookout doctor

Quick start

# zero-config: point it at any local app
lookout targets --url http://localhost:3000

# per-repo setup
lookout init                  # scaffolds .lookout/config.ts
lookout targets               # resolve + probe the configured targets

Verbs

| verb | what it does | | --------- | ------------ | | capture | screenshots + deterministic findings (console errors, overflow, axe), no AI | | check | capture + AI judge against the base rubric plus the project rubric; findings merge into .lookout/backlog.json | | verify-fix | rule on a claimed fix: re-capture and re-judge one issue's routes, then close it or leave it open with a note on what the judge still sees | | verify | judge the app against acceptance criteria (--criteria ticket.md or inline text); per-criterion pass / fail / not-visually-verifiable with evidence | | ask | answer a free-form question about the rendered app, grounded in fresh screenshots | | backlog | adjudicate findings: merge, set statuses (fixed / by-design / blocked, with mandatory reasons), regenerate the report, check for staleness | | targets | list configured targets and probe reachability | | skills | lookout's own instructions: list, diff, freeze, replay, improve | | self-heal | fix what lookout keeps getting wrong, in lookout's own source | | init | scaffold .lookout/config.ts | | status | what the run in flight is doing, folded out of the event log; exit 1 while a run is going, so an agent can poll it | | ui | a local page rendering that same log live, with thumbnails, findings and verdicts, for a person to watch | | protocol| the operating contract, printed by lookout itself, for whichever agent is driving it | | doctor | verify prerequisites |

Exit codes: 0 clean, 1 findings / failed criteria, 2 execution error, so agents and CI can gate on the result. verify-fix adds 3 for an issue blocked after exhausting its attempts. Every verb takes --json for machine-readable output.

Watching a run

A run takes minutes, and a subprocess's stdout does not reach its caller until it exits. So lookout narrates to .lookout/evidence/events.jsonl as it goes, and two readers render it while the run is still going:

lookout status          # for an agent: phase, shots, findings, issues
lookout ui              # for a person: the same log as a live local page

Both are readers. Either can watch a run started by anything, in any terminal.

capture, check and verify-fix also composite every shot into one labelled contact sheet, a view's dark and light captures side by side and defect-carrying tiles marked, so a session can see what lookout saw for the cost of one read. Full-resolution paths are printed beside it for close reading.

Issues

Open findings group into issues by root cause rather than by screenshot: one target + category + attribute, so a theme that never switches is one issue across every route it spoils rather than one per shot. Co-located accessibility violations group by route instead, because an axe rule id names the rule that fired rather than the thing that is wrong, and one malformed widget trips several at once.

Each issue gets a six-digit id and a folder under .lookout/issues/<id>/, and Issue.md in that folder is the self-contained document a fix session is handed: what is wrong and where, the screenshots (copied into img/ and listed by absolute path), the acceptance criteria it will be graded against, and what lookout has ruled so far. lookout writes the folders and rules on the outcomes; who fixes an issue, and how, is not lookout's call.

When a fix is claimed, lookout rules on the claim:

lookout verify-fix --issue <id> --commit <sha> --note "<root cause>"

That re-captures and re-judges only that issue's routes. Exit 0 means the defect is gone and the backlog is adjudicated to fixed with the commit. Exit 1 means it is not: the finding stays open, with a note on what the judge sees now. Exit 3 means the issue exhausted --max-attempts (default 2) and is recorded as blocked with a reason. A fix session never grades its own work.

Safety defaults

• Targets must be localhost unless --allow-remote is passed explicitly.
• lookout never starts services; a down target prints the project's startHint and exits.
• Judging runs claude -p with read-only tool access, cwd-pinned to the evidence directory.

Per-project config

.lookout/config.ts is a TypeScript module (the CLI runs under bun, so recipes are real functions) default-exporting a LookoutConfig. Everything below is optional except targets.

import type { LookoutConfig } from "@nannier-com/lookout";
import type { Page } from "playwright";

const config: LookoutConfig = {
  project: "myapp",

  targets: [
    {
      name: "app",                       // handle for --targets and fingerprints
      url: "http://localhost:3000",
      startHint: "bun run dev",          // printed when down; lookout never starts services
      readyPath: "/",                    // polled for reachability
      routes: [
        "/",                             // string shorthand
        { path: "/checkout", name: "Checkout", states: ["cart-open"] },
        { path: "/settings", element: "main" },  // element screenshot instead of full page
      ],
    },
  ],

  // Viewport presets (desktop-first defaults: 1440x900 / 834x1112 / 390x844).
  viewports: { phone: { width: 375, height: 812 } },

  // How the app switches dark/light:
  //   emulate (default)  prefers-color-scheme emulation
  //   url-param          lookout appends ?<param>=dark|light to every route
  //   recipe             this module also exports setScheme(page, scheme)
  scheme: { mode: "url-param", param: "scheme" },

  // Named interaction recipes. A route opts in via states: ["name"]; each
  // state is captured at every requested form factor and scheme, right after
  // prepare() returns. restore() puts the page back; without it lookout
  // reloads between states.
  states: {
    "cart-open": {
      prepare: async (page: Page) => {
        await page.getByRole("button", { name: "Cart" }).click();
        await page.getByRole("dialog").waitFor();
      },
      restore: async (page: Page) => {
        await page.keyboard.press("Escape");
      },
    },
  },

  // Project judging rules, layered into the visual-judge skill in every judge
  // prompt. Editing this file re-judges whatever it could have changed: the
  // ledger is keyed on the composed prompt itself, so nothing has to be bumped
  // by hand. An optional `rubricVersion: N` header is still read, and shows up
  // in the key for anyone reading ledger.json.
  rubric: "./rubric.md",

  // One-line suppressions for things the base rubric would flag but this
  // project does on purpose.
  neverFile: ["the marketing hero intentionally overflows on phone"],

  // Native apps (capture with --platforms ios,android). Both schemes on a
  // device need appearanceParam: the app must read the scheme from the deep
  // link, because OS-level appearance flips cannot reach apps that manage
  // their own theme.
  native: {
    target: "app",
    ios: { deepLinkScheme: "myapp", bundleId: "com.example.myapp" },
    android: { deepLinkScheme: "myapp", bundleId: "com.example.myapp", settleMs: 14000 },
  },
};

export default config;

Skills: where lookout's AI behaviour lives

Every AI capability is an instruction file, not a string in the binary. They ship in the standard Agent Skill layout, one directory each:

skills/
  visual-judge/       SKILL.md + rubric.md: what counts as a defect, and how to file it
                      handoff.md: comparing against a design hand-off, carried
                      only when a shot in the batch has one
  refute-finding/     SKILL.md: the adversarial pass that kills false findings
  verify-acceptance/  SKILL.md: ruling on a ticket's criteria from evidence alone
  fact-check/         SKILL.md: answering one question from screenshots

A skill carries the whole prompt shape, placeholders and all. lookout supplies only data: the shot manifest, the paths, the question. Each declares {{amendments}}, the slot where a project's own layer lands.

Every judge prompt is therefore assembled from the visual-judge skill (severity ladder, the closed category vocabulary, the judging procedure, the universal never-file list), then this project's .lookout/skills/visual-judge/ layer if it has one, then its rubric file, then its neverFile lines. Findings outside the category vocabulary are rejected at ingestion, so project extensions refine judgment; they cannot invent new taxonomies.

The ledger is keyed on the composed prompt itself, judging and refuting instructions together, so amending a skill, a project rubric or a neverFile line invalidates exactly the cached verdicts it could have changed, and nothing else. That covers the refuting skill too, because what the ledger stores is what survived it.

What the judge rules on

The rubric sorts everything it could say into three bands, because the useful question is not "is this good" but "is this mine to call".

Execution defects are filed without argument: broken, illegible, overlapping, clipped, unrendered.

Design quality is filed too, on one condition: the finding must name the principle it breaks and what that costs the person using the screen. "The card's title, metadata and body are all one size and weight, so the eye has no entry point" is a finding; "the card looks bad" is not. This is the band a model is actually strongest in, because it is a judgment about the whole rather than a measurement, and the citation requirement is what keeps it falsifiable enough to verify and to write acceptance criteria for.

Product and brand decisions are left alone: which blue, which typeface, how round the corners, how dense the information, the voice of the copy. lookout rules on what a decision does in context (a brand colour that leaves text unreadable is a contrast defect) and never on the decision.

Two things follow from this that are worth knowing. The judge is told it cannot measure, because it is reading an image: it files a geometry finding only when the deviation is visible without looking for it, and never quotes a pixel value it did not read off the screen. And it is told not to rule on anything a still image cannot show, such as focus order, which is what the deterministic axe pass and the console checks are for.

Skills that improve themselves

lookout skills list               # what lookout knows how to judge, and what this project has amended
lookout skills freeze             # freeze the settled verdicts into a regression set
lookout skills improve            # learn from this project's runs, and keep it only if the set holds
lookout skills replay             # judge the frozen set with the skills as they stand

improve reads signals lookout already records: findings the adversarial verifier refuted, findings a person adjudicated by-design and wrote a reason for, defects that survived every attempt, acceptance criteria that could not be decided from a screenshot, and replies that failed the output contract. It writes the amendment that would have prevented the most of them into this project's layer.

It applies automatically, and what makes that safe is the gate. .lookout/regression/ holds screenshots whose verdicts were settled when the pixels were fresh: what a person ruled intentional, and what the verifier confirmed. The candidate is replayed over them, and an amendment that re-files a suppressed finding or loses a confirmed one is rolled back, with the attempt and its violations written to .lookout/skills/history.jsonl. An amendment nothing can grade, because the set is empty or its pixels are not on this machine, is written to PROPOSED.md instead of being applied.

The manifest is committed and the frozen pixels are not; lookout skills freeze rebuilds them from the evidence store.

Healing itself

lookout self-heal --project ~/code/some-app

Failures lookout hits are appended to ~/.lookout/incidents.jsonl, pooled across every project on the machine and never truncated: crashes, operator errors, judge replies that could not be parsed, findings rejected at ingestion. self-heal groups them, fixes the cause of one group in lookout's own checkout, and is not believed about any of it.

The subprocess may read and edit inside the checkout and may not run a single command. lookout runs tsc --noEmit, eslint, bun test and the build itself, plus a replay of that project's frozen regression set when --project names one. Any gate failing reverts everything, keeps the diff and the gate output under ~/.lookout/self-heal/<stamp>/, and records the rollback as an incident. Every gate passing commits the change alone with a patch changeset, and does not push: a local commit is one git revert away.

It refuses to run on an installed package (no source, no repository), over a dirty checkout, or while another heal holds the lock.

Where things land

.lookout/
  config.ts        committed: the project's targets and recipes
  backlog.json     committed: adjudicated findings (managed via `lookout backlog`)
  BACKLOG.md       committed: generated report (regen via `lookout backlog regen`)
  ledger.json      committed if you want cross-machine judge caching
  issues/<id>/     committed: one folder per issue, named by its six-digit id
                     Issue.json, Issue.md, state.json committed;
                     img/ gitignored with the evidence
  skills/          committed: this project's layer over lookout's shipped skills
  evidence/        gitignored: screenshots + capture-report.json + judge-report.json

The fix loop (for agents)

lookout never edits code. The loop it is built for:

1. lookout check in the target repo: findings merge into the backlog.
2. Fix the code in that repo, per that repo's own conventions.
3. lookout check --targets x --routes /y to re-capture and re-judge just the affected scope; unchanged pixels stay ledger-cached.
4. lookout verify-fix --issue <id> --commit <sha> --note "<root cause>" is the only thing that closes a finding. It rules on that issue alone: a fix that clears the defect and causes another one passes, and the new defect is filed as its own numbered issue with a note saying which fix surfaced it.
5. lookout backlog set <fingerprint> --status fixed --commit <sha>; use --status by-design --reason "..." for intended behavior (suppressed in every later merge) and --status blocked --reason "..." after repeated failed attempts.
6. lookout backlog check as the gate: schema, mandatory reasons, missing issue ids, markdown freshness, and drift detection all fail loud.