mm-harness

One CLI for operating MetaMask Extension, Mobile, and Core and producing reviewable recipe evidence. Run it inside a checkout; the product, slot, ports, and runtime paths are detected automatically.

• Action: one typed operation.
• Recipe: a reusable, parameterized graph of actions and called recipes.

The generic graph engine and evidence schemas live in Farmslot packages. mm-harness owns MetaMask runtime control and domain capabilities.

Daily and release QA

The same recipes can validate yesterday's domain changes together or a release that is behind main. The mms-recipe-qa skill freezes the change scope and criteria; the team library supplies smoke and reusable journeys; this CLI runs them on the selected app and retains assertions, traces and captures.

Use the installed skill's references/domain-qa.md for window/range inputs and report policy. Scope selection and scheduling are outside the harness. Discover recipes through run --list and run <recipe> --describe, then follow the locked version's help --json for composition and drift diagnosis. Adapt a recipe to the tested release without changing the release or weakening its criteria. An internal action that needs development instrumentation cannot prove behavior on an unmodified release artifact.

Getting started from zero

No checkouts yet? mm-harness setup-base clones the MetaMask product repos into one standard layout and runs each repo's own dependency install — no install of this package required to start:

npx -p @deeeed/metamask-harness mm-harness setup-base

It produces, under ~/dev/metamask by default:

metamask-extension-1..N   metamask-mobile-1..N   core-1..N

Run without flags in a terminal, it asks which projects you want and how many copies of each. Pass flags to skip the prompt entirely — which is how scripts, CI, and parent tools should invoke it:

mm-harness setup-base --only core --counts core=1
mm-harness setup-base --dir ~/work/metamask --counts extension=3,mobile=1,core=2
mm-harness setup-base --only mobile --dry-run        # plan only, changes nothing

The base directory resolves by precedence: --dir, then $MM_HARNESS_BASE_DIR, then saved preferences, then ~/dev/metamask. A successful run saves the base directory and counts it used, so later runs adopt them — inspect with --show-config, clear with --reset-config. Re-running is safe: existing clones are fetched, never reset and never deleted.

Scope fence. It clones and installs dependencies. Nothing else — no platform toolchains, no simulators, no .env files, no builds. Workflow onboarding is a prompt; environment readiness is mm-harness doctor, which is exactly where it points you next.

setup-base is a normal command on the single mm-harness entrypoint. Its work stays in one readable shell leaf, and npx -p makes that same command available before a global install.

Install

npm install -g @deeeed/metamask-harness@latest
mm-harness --version
mm-harness doctor

doctor is read-only. doctor --fix repairs harness-owned runtime state but does not launch an app, invent credentials, or choose a wallet fixture.

mm-harness doctor --fix
mm-harness fixtures init --from /secure/path/wallet-fixture.json
# Disposable public testing only; never fund this wallet:
mm-harness fixtures init --dev

Use mm-harness update to update a published installation.

Operate

# Extension
mm-harness launch
mm-harness launch --sidepanel

# Mobile
mm-harness launch ios
mm-harness launch android

# Any checkout
mm-harness status
mm-harness stop
mm-harness logs
mm-harness debug
mm-harness reload                 # Mobile Metro reload / Extension CDP refresh
mm-harness fixtures set

# Delete the current wallet, then reapply the canonical fixture from clean state.
mm-harness fixtures reset

Extension launch keeps its incremental watcher running. Refresh the active page after a successful rebuild; use launch --build only for an explicit production-like LavaMoat rebuild.

Extension browser

RECIPE_HARNESS_BROWSER picks the browser that hosts the unpacked extension:

| Value | Browser | |---|---| | auto (default) | The checkout's Playwright Chrome for Testing when it starts and paints on this machine, else Google Chrome | | cft | Chrome for Testing, never probed: the behaviour before auto existed | | chrome | Google Chrome | | /abs/path | That executable |

RECIPE_HARNESS_CHROME_BIN=/path still overrides everything and is used as given. Google Chrome ignores --load-extension, so the harness loads the build over CDP (Extensions.loadUnpacked, with --enable-unsafe-extension-debugging) into the slot's own profile and checks the id against the manifest key. Other user extensions in that profile are disabled, as --disable-extensions-except does for Chrome for Testing; extensions an enterprise policy pins cannot be disabled, so the launch warns and records them with the run evidence.

In auto, a launch with a new profile probe-launches Chrome for Testing the way the launcher starts it (headful, through Launch Services on macOS). The verdict lives in ~/.cache/mm-harness/browser-probe.json: a crash is kept until the OS build or browser changes; a probe that only timed out is retried after 30 minutes. Concurrent launches wait for one probe per browser build (a <cache>.<key>.locks/ directory of numbered tickets next to the cache; a holder older than 3 minutes counts as gone, so a probe that outlives it can overlap the next one, and during an upgrade from a build that used a .lock file one duplicate probe is possible). Tickets are never removed by the harness: each probe adds two small files, and a crashed holder's unreleased ticket stays (it no longer blocks once its process is gone). To reclaim the space, remove the lock directories while no launch or doctor is running: rm -r ~/.cache/mm-harness/browser-probe.json.*.locks. On a cold cache the probe can take up to about 80 seconds and shows a small (about 480×360) browser window; doctor --adapter extension runs the same probe.

On a Mac whose Google Chrome is managed by policy (force-installed extensions in /Library/Managed Preferences/…/com.google.Chrome.plist, or forced sign-in), auto uses Chrome for Testing: it does not read those policies, so no forced extension can open windows in the slot. If Chrome for Testing does not start and paint there, the launch stops and names the two ways forward (RECIPE_HARNESS_BROWSER=chrome, accepting the forced extensions' windows, or a less loaded machine) instead of falling back to the managed Chrome. Doctor lists the forced extensions. A slot profile created on the managed Chrome keeps it, with a warning to reset the profile.

A slot then keeps the kind of browser its profile was created with, until the profile is reset (mm-harness fixtures set or fixtures reset):

• created on Chrome for Testing: the checkout's current Chrome for Testing (a Playwright upgrade is followed and re-probed); it moves to Google Chrome only once that build is proven to crash, never back;
• created on Google Chrome or an explicit path: that binary; if it is removed, the launch fails and says how to recover rather than switching browsers;
• created with RECIPE_HARNESS_CHROME_BIN, a path in RECIPE_HARNESS_BROWSER, or a launcher given --chrome-bin directly: that exact binary on every later auto launch, even when it is a Chrome for Testing build.

A preserved slot stuck on Chrome for Testing that "renders no frames" (a transient verdict, which keeps it on Chrome for Testing) moves to Google Chrome without a profile reset: launch it once with RECIPE_HARNESS_BROWSER=chrome. The profile is then a Google Chrome profile and later auto launches keep Google Chrome. Going the other way opens a profile in an older browser, so reset the profile instead.

Profiles created before this browser record existed have no record, so their first auto launch probes like a new profile and may move to Google Chrome.

mm-harness doctor --adapter extension shows the current choice; each launch writes it to temp/recipe/runtime/browser-resolution.json with the browser process identity, and run evidence (product-provenance.json) names the browser the run drove only after checking that exact process still serves the CDP port the run executed on. The owner of a CDP port is the process named by the profile's SingletonLock, and only if the lock names this host, the process started before the lock was written (a reused pid did not), and it holds files open in the profile. That proof uses lsof, which macOS ships; without it the launch stops with an explanation. Start times are compared exactly as ps -o lstart prints them in UTC, so a record written by an earlier harness build (local time) binds nothing (boundTo: none) until the slot relaunches.

On macOS the slot browser starts in the background with no startup window, and its first window opens over CDP as a background window. After the launch settles there is one focus check: only if the slot browser's own process is frontmost does the previously focused app get the front back, once (MM_HARNESS_FOCUS_HOLD=0 turns this off). A click on the slot window right after launch can therefore be undone once.

# Roll back to Chrome for Testing with a fresh slot profile (a profile Google
# Chrome has opened is not reopened in the older Chrome for Testing). Keep
# RECIPE_HARNESS_BROWSER=cft set for later launches too: on a plain auto launch
# the profile follows Chrome for Testing only until a probe proves it dies.
RECIPE_HARNESS_BROWSER=cft mm-harness fixtures set
rm ~/.cache/mm-harness/browser-probe.json        # forget probe verdicts

Remote feature flags

--remote-flag KEY=VALUE[,KEY=VALUE] pins a remote feature flag for the launch, which is how a screen gated behind an A/B variant becomes reachable:

mm-harness launch --remote-flag perpsTAT3938AbtestScreenVsBottomSheet=treatment
mm-harness launch ios --remote-flag perpsEnabled=true,perpsConfig='{"tier":2}'

A bare word is a named variant (treatment becomes { "name": "treatment" }), true/false is a boolean, and a value opening with { or [ is parsed as JSON. The grammar is parsed once and every leaf receives the resolved map, so a boolean stays a boolean rather than becoming the string "true".

Every launch makes the app's override set exactly the wallet fixture's remoteFeatureFlags object (the baseline every run shares) merged with the --remote-flag map (a one-launch override), on both clients. A plain launch therefore drops a previous run's --remote-flag variant and keeps the fixture's. Mobile clears and re-applies the set through the running RemoteFeatureFlagController and fails the launch if it does not read back. The Extension writes it into the per-run dist snapshot manifest and re-reads the harness-owned record there (_harness.remoteFeatureFlags, written on every snapshot), so the launch reports the pin it proved; reuse of a running browser is allowed only while that record already equals the request, a reload in place carries the record across the rebuilt snapshot, and --watch (which never re-creates the snapshot) refuses --remote-flag. A recipe's own recovery relaunch carries the live pin rather than dropping it mid-run. A verified release artifact is never rewritten, so a launch that would have to pin it (--remote-flag or a fixture that declares flags) refuses instead of testing the wrong arm.

Read or change flags on a running app with the actions. fixtures set follows the same rule as launch: the fixture's block is the baseline and fixtures set --remote-flag KEY=VALUE layers a one-run override on it, which is how prepare --remote-flag keeps its pin through the fixtures step.

mm-harness call metamask.feature_flags.read
mm-harness call metamask.feature_flags.set --arg 'flags={"perpsEnabled":true}'   # mobile
mm-harness call metamask.feature_flags.clear                                      # mobile

doctor and status print a feature flags: line naming every active override, so a slot never runs a proof under an unnamed variant. The line distinguishes (no runtime) and an unreadable snapshot from a genuine 0 override(s).

Log sources stay separate:

mm-harness logs --source extension  # Extension page
mm-harness logs --source dapp       # active dapp
mm-harness logs --source webpack    # compiler
mm-harness logs --source watcher    # watcher lifecycle
mm-harness logs --source rebuild    # incremental rebuilds
mm-harness logs --source app        # Mobile app
mm-harness logs --source metro      # Mobile bundler

Live Mobile and Extension recipes automatically index a bounded, redacted network/run-summary.json with request timing and recipe-node boundaries. Use explicit app.network_capture and app.network_assert nodes for filtered windows and machine-checked request expectations. See Recipe-scoped network capture.

Mobile and Extension recipes can bracket explicit UI-smoothness windows with app.performance_capture and verify their evidence with app.performance_assert. Capture is never automatic. See UI smoothness capture.

Work a task

# Pick a checklist from the shared catalog.
mm-harness execution-template list --package-templates <catalog> --json

# Write the task directory the agent works from.
mm-harness task init temp/tasks/fix/TAT-1 \
  --flow fix-bug --template fix-bug/mobile \
  --package-templates <catalog> --title "Wrong total on the receipt" --ticket TAT-1

# Bring the checkout to a proven-ready state before the first step.
mm-harness prepare --artifacts-dir temp/tasks/fix/TAT-1/artifacts --json

# Mark progress; the task's own ./mark shim routes here.
mm-harness checklist mark temp/tasks/fix/TAT-1 start
mm-harness checklist mark temp/tasks/fix/TAT-1 1
mm-harness checklist mark temp/tasks/fix/TAT-1 complete --mark-last

task init writes TASK.md, CHECKLIST.md, the mark shim, and inputs/. CHECKLIST.md is the only file whose boxes count as steps. It writes no checklist-target.json; absent means CHECKLIST.md + SIGNAL.json.

prepare runs doctor --fix, status, launch --verify, fixtures set, and verify in order, stops at the first failure, and records one readiness record at <artifacts-dir>/sandbox.json. Core skips launch and fixtures; Mobile takes ios/android from the checkout's configured target, then from the devices status reports, and asks when that is ambiguous. Exit 0 means ready. See the task directory.

Discover and prove

mm-harness actions positions
mm-harness actions --action metamask.wallet.ensure_unlocked
mm-harness call metamask.wallet.ensure_unlocked

mm-harness run --list
mm-harness run perps.smoke --describe
mm-harness run wallet.smoke --describe
mm-harness run wallet.import method=auto
# When a visible Terms of Use gate follows import and acceptance is authorized:
mm-harness run wallet.accept-terms accept=true

# Use visible client UI to delete and import the wallet again.
mm-harness run wallet.reset-import
mm-harness run path/to/recipe.json market=ETH --plan
mm-harness run path/to/recipe.json

run selects a checkout-local artifact directory unless --artifacts-dir <dir> overrides it. Human output prints diagnostics and absolute paths to the report, trace, executed recipe, and artifact manifest. Use mm-harness last --json to resume without repeating the last operation.

For automation, --json emits one stable document. run --json-stream emits line-buffered JSONL progress and a terminal event. Its status matches the recipe's pass, fail, or unknown verdict; unknown still exits nonzero.

Team libraries

export RECIPE_LIBRARY_PATH="team=$HOME/shared-library/team-recipes"
mm-harness run --list

Shared libraries hold durable actions and composable recipes. Task acceptance criteria remain task-local. See Recipes.

For optional model advice matching acceptance criteria to library flows and assessing drafts or evidence, see Optional recipe advisor. Select a provider with --advisor typesafe or save advisor.enabled and advisor.provider through mm-harness config. Inspect the effective policy with mm-harness recipe-quality advisor-status --json; use --advisor off per task. Ordinary recipe execution needs no advisor or API key.

Exporting is not required: a library registered with mm-harness config set libraries.<name> <path>, or checked out beside the product checkout, reaches run, actions, and template discovery too. mm-harness status lists what this machine resolves, with each location's revision and recipe count.

Teams maintain their library checkouts. Update them before starting a task; the harness records local revisions and recipe digests but does not fetch or check for upstream updates. Keep the selected library unchanged during a run.

Publish a change

# Write artifacts/pr-description.md in the repository PR template shape (prose
# only), then render the publishable body.
mm-harness pr-body render temp/tasks/dev/TAT-1 --command 'mm-harness run artifacts/recipe.json'

pr-body render writes artifacts/pr-body.md: the description with the "Validation Recipe" and "Validation Logs" sections rendered from artifacts/recipe.json and artifacts/recipe-run/report.md (and "Recipe Workflow" from artifacts/workflow.mmd when it exists), replacing any pasted by hand. The worker never pastes artifacts into markdown, so fences cannot break.

Review a change

# Which team library owns the change (declared --domain wins, else owned-paths.json).
mm-harness domain

# The review guide: base review + the library's anti-pattern families and parity rule.
mm-harness help review --domain perps

# A checklist the mark shim can track: Setup, Base review, Domain patterns, Parity, Verdict.
mm-harness review checklist --domain perps --out temp/tasks/review/TAT-1/CHECKLIST.md
mm-harness review checklist --domain perps --since <last-reviewed-sha>   # incremental re-review

# Where this machine keeps libraries and the other platform's checkout (outside any repo).
mm-harness config set references.extension ~/dev/metamask-extension

A team library supplies review/antipatterns.md (one ## section per family), optional review/antipatterns.<adapter>.md families appended for that adapter only, review/parity.md, and owned-paths.json; the harness composes them and never copies them. Locations resolve env (RECIPE_LIBRARY_PATH, MM_HARNESS_REF_<ADAPTER>) → mm-harness config → sibling directory beside the checkout → shallow fetch into <checkout>/.skills-cache/<name>. A missing reference checkout degrades the parity step to "not checked" with the reason.

Recover

mm-harness doctor
mm-harness doctor --fix
mm-harness verify
mm-harness cleanup

Failures name one exact next action. Core is headless; browser, device, logs, and debugger capabilities are reported as unavailable instead of fabricated. Stable exit codes are 1 runtime/action failure, 2 invalid CLI usage, 3 infrastructure failure, 4 bounded recovery refusal, and 5 validation/trust failure.

Reference

• Recipes — discover, compose, author, and share proof.
• Security — trust, approval, fixtures, and evidence safety.
• QA — clean-machine and human release checks.
• Contributing — ownership, layout, and change gates.

For development, point the installed command at a source checkout:

export MM_HARNESS_BIN=/path/to/metamask-harness/bin/mm-harness
mm-harness --version

Unset MM_HARNESS_BIN to return to the published installation.

Workflow entry points and completion

Explicitly invoke mms-recipe-dev, mms-recipe-fix-bug, mms-recipe-qa or the team's static review skill. mms-recipe-cook selects or resumes their shared Markdown checklist. Matching descriptions or changed paths never activates these workflows.

Static Perps review uses a generated source checklist without installing or launching the app harness. recipe-qa freezes a PR, range or daily change window and requires runtime smoke plus selected recipe proof. Official release validation uses its named release profile and proof lane.

TASK.md holds the request; CHECKLIST.md is the sole step list. The frozen terminal contract and task-local mark validate completion. Publication, retained sessions and workspace cleanup belong to the caller. Farmslot handles them through its control plane; direct invocation returns the outcome to the engineer. See the task-directory guide and QA tutorial.

Portable runtime proof

validateRuntimeProof(bundle, smokeTarget) validates a retained Recipe v1 artifact package, requiring executed behavioral assertions for its declared targets. A passing command or end node alone is insufficient. QA skills use this exported function through their locked harness installation; no gateway, farm profile or run ID is required.