still-qa

still-qa is a pre-render QA gate for AI-generated image sets. Point it at a folder of stills and it flags defects that generators produce silently and that humans only catch after the video is assembled: fully black frames, baked-in letterbox bars, and aspect-ratio drift. It emits one JSON report and one exit code, so it drops into a pipeline before the expensive step. The dead-frame and letterbox thresholds are not guesses — they were derived by measuring real shipped stills alongside the defective ones caught against them, and the derivation is in docs/calibration.md so you can re-run it against your own model's output. still-qa is not a dedup tool: for dataset-scale near-duplicate detection, use idealo/imagededup instead — still-qa's job is the pre-render gate, not deduplication.

Requirements

• Bun ≥ 1.0
• ffmpeg on PATH (used for luminance measurement)

Usage

bunx still-qa ./shots --aspect 16:9 --out report.json

usage: still-qa <dir-or-glob> [options]

  --aspect W:H         expected aspect ratio (default: infer the modal ratio of the set)
  --aspect-tol N       max |w/h - expected| deviation (default 0.01)
  --dead-yavg N        dead-frame threshold, whole-frame mean luminance (default 2)
  --bar-yavg N         letterbox threshold, per-band mean luminance (default 15)
  --band-fraction N    top/bottom band height as a fraction of frame height (default 0.06)
  --only a,b           run only these checks (dead, letterbox, aspect)
  --skip a,b           skip these checks
  --measure-only       print measured values, no verdicts, exit 0
  --format json|text   output format (default json)
  --out FILE           write the report to FILE instead of stdout

Exit codes: 0 scanned, everything clean · 1 scanned, at least one still failed a check · 2 usage/tool error. That contract is the CI integration — no plugin needed:

still-qa ./shots --aspect 16:9 || exit 1

The checks

| Check | What it catches | How | |---|---|---| | dead | Fully black frames some generators return stochastically | whole-frame mean luminance < --dead-yavg (default 2) | | letterbox | Baked-in cinematic bars that no longer match the timeline | top and bottom bands (default 6% of height each) both < --bar-yavg (default 15). Both bands, never one: real keepers legitimately have a single near-black band (night sky, dark ground) | | aspect | Wrong-dimension output in an otherwise uniform set | |w/h − expected| > --aspect-tol; with --aspect omitted, the modal ratio of the set is inferred and the minority flagged |

A file that cannot be measured (missing, not a regular file, decode/ffmpeg error) fails closed with reason unmeasurable and drives exit 1 — it never silently passes. (Fail-closed enumeration of non-regular files applies to directory scans; a glob target only yields what the glob engine returns.)

The report

Machine-readable JSON is the primary surface (--format text is the human summary). The effective thresholds — whatever their source — are always echoed back:

{
  "version": 1,
  "tool": "[email protected]",
  "thresholds": { "deadYavg": 2, "barYavg": 15, "bandFraction": 0.06, "aspect": "16:9", "aspectTol": 0.01, "checks": ["dead", "letterbox", "aspect"] },
  "files": [
    { "path": "shots/s003.png", "width": 1280, "height": 720, "yavg": 0.0000955, "bandTop": 0.4, "bandBottom": 0.3, "fail": ["dead"] }
  ],
  "summary": { "scanned": 40, "failed": 3, "byCheck": { "dead": 1, "letterbox": 1, "aspect": 0, "unmeasurable": 1 } }
}

version is the report-shape contract; new fields (e.g. near-duplicate pairs, planned for v0.2) will bump it.

Configuration

Precedence: built-in defaults < still-qa.json in the scanned directory < flags. A still-qa.json can pin per-set thresholds next to the images:

{ "barYavg": 12, "aspect": "16:9" }

Calibration — the defaults are model-specific

The shipped defaults were calibrated on 1280×720 flux/schnell output. A different model's dark scenes or a different resolution can misfire — that's the failure mode that makes people uninstall a QA tool, so every threshold is a flag, the report echoes the effective values, and --measure-only prints raw measurements with no verdicts so you can derive your own numbers in one command:

still-qa ./my-keepers --measure-only --out keepers.json
still-qa ./my-rejects --measure-only --out rejects.json

Measure your keepers, measure your defects, put the threshold in the gap. The full recipe and the original derivation are in docs/calibration.md.

Library use

import { checkDir } from "still-qa";

const report = await checkDir("./shots", { aspect: "16:9" });
if (report.summary.failed > 0) throw new Error("defective stills — not rendering");

Development

bun install
bun run test    # generates synthetic fixtures, then runs the suite

All test fixtures are synthetic, generated deterministically by test/fixtures/gen-fixtures.ts — nothing in the suite depends on private image sets. Each check's tests were written first and verified to fail against a stub implementation before the real one landed.

Decisions made during build

• Directory targets scan top-level PNG/JPEG entries only (non-recursive); use a glob for anything else.
• still-qa.json is read only for directory targets (from the scanned directory itself). Glob targets use flags/defaults only — a config file in the process cwd never reconfigures a glob scan of somewhere else.
• --only/--skip combinations that leave zero checks are a usage error (exit 2), never a green no-op scan. --band-fraction must be strictly between 0 and 0.5.
• --measure-only always exits 0 (unless usage error) — it renders no verdicts, so it reports no failures.
• With --aspect omitted, the report echoes the inferred ratio as e.g. "16:9 (inferred)".
• Generated fixtures are gitignored; the committed generator is the fixture of record (a FIFO can't be committed anyway).
• The repo sits at version 0.0.0 until the first real release is published.

License

MIT