UpgradeGuard

Find Android bugs that only happen when existing users update your app.

Your new APK may work perfectly from a clean installation while breaking for users upgrading from the previous version.

UpgradeGuard automates that upgrade path.

npx upgradeguard test

UpgradeGuard

Testing upgrade

1.0.0 (1)
     ↓
1.1.0 (2)

────────────────────────────

ENVIRONMENT

✓ ADB
✓ APK reader
✓ Maestro
✓ Android device
✓ JAVA_HOME


APK VALIDATION

✓ Same package
✓ Version transition valid


PRE-UPGRADE

✓ Previous version installed
✓ Preparation flow passed


UPGRADE

✓ New APK installed preserving data
✓ Application launched


POST-UPGRADE

✗ Verification flow failed

Expected:
id: home-screen is visible


RUNTIME

✓ No native crash
✓ No ANR detected (best-effort — ANR log markers are unverified)

────────────────────────────

RESULT

❌ UNSAFE UPGRADE

1 regression detected

Demo recording coming soon.

Problem

A clean install and an upgrade are different code paths, and only one of them is normally tested. On a clean install the app starts from empty storage. On an upgrade it starts from storage that a previous version wrote, under the keys and schemas that version chose.

Renaming a persisted key, changing a stored shape, or forgetting a migration produces a build that is internally consistent and passes every clean-install test, while sending every existing user back to onboarding, to a login screen, or to a crash.

That class of bug reaches production because reproducing it by hand is tedious: install the old build, use the app until it has real state, install the new build over it without wiping data, and check the app still works. UpgradeGuard runs that sequence.

How it works

1. Reads upgradeguard.yml and checks the environment first (adb, an APK reader, Maestro, a booted and unlocked device, JAVA_HOME). Nothing is installed until this passes.
2. Reads both APKs and validates the transition: same package name, non-decreasing versionCode, and matching signer when apksigner can be located.
3. Installs the old APK with adb install -t.
4. Runs the flows.before Maestro flows to create real user state — sign in, complete onboarding, whatever your app persists.
5. Upgrades in place with adb install -r -t, which keeps the application's data directory.
6. Launches the app and runs the flows.after Maestro flows against the upgraded install.
7. Reads logcat for native crashes and, best-effort, for ANRs.
8. Prints a report and exits 0, 1 or 2.

Maestro's verdict comes from its JUnit XML output and from preflight checks, never from its exit code, which is the same 1 for an assertion failure, a missing flow file and "no device connected". A tool problem exits 2, not 1: "we could not test this" is never reported as "your upgrade is broken".

Requirements

• Node.js 24 or newer.
• A JDK 17 or newer on PATH, with JAVA_HOME set. apkanalyzer, apksigner and Maestro all need it.
• Android SDK platform-tools, for adb.
• An APK reader: either apkanalyzer (ships with the SDK's cmdline-tools) or aapt2 (ships with build-tools). Either one is enough; apkanalyzer is preferred when both exist. Neither has to be on PATH — UpgradeGuard also looks inside $ANDROID_HOME/build-tools/<version>/.
• Maestro. On Windows there is no npm, winget or Chocolatey package: it is a manual ~315 MB zip download plus a PATH edit. upgradeguard doctor prints the exact steps.
• A device or emulator that is booted and unlocked. A freshly booted emulator reports sys.boot_completed=1 while user 0 is still credential-locked, for roughly 90 seconds. During that window Maestro cannot install its driver (SecurityException: Package dev.mobile.maestro is not encryption aware!) and then waits indefinitely rather than timing out — MAESTRO_DRIVER_STARTUP_TIMEOUT is not honoured there on Windows. UpgradeGuard checks for this and tells you to wait, instead of hanging.

Run upgradeguard doctor to check all of the above and get install instructions for whatever is missing.

Installation

No install needed:

npx upgradeguard test

Or add it to the project:

npm install --save-dev upgradeguard

Or install it globally:

npm install -g upgradeguard

Quick start

1. Get two APKs of the same app, signed with the same key: the version your users have, and the version you are about to ship.
2. Write two Maestro flows — one that creates user state, one that verifies the app after the upgrade.
3. Create upgradeguard.yml next to them:

app:
  package: com.example.app

upgrade:
  from: builds/app-1.0.apk
  to: builds/app-1.1.apk

flows:
  before:
    - flows/prepare-user.yaml
  after:
    - flows/verify-upgrade.yaml

4. Boot a device or emulator, unlock it, and run:

npx upgradeguard test

A complete, runnable example lives in examples/demo-app/ in this repository.

Configuration

upgradeguard.yml, read from the current directory unless --config says otherwise. Paths are resolved relative to the configuration file, not to the working directory.

app:
  package: com.example.app # optional; asserted against the old APK, never used in place of it

upgrade:
  from: builds/app-1.0.apk # required — the version already on users' devices
  to: builds/app-1.1.apk # required — the version being shipped

flows:
  before: # default: []
    - flows/prepare-user.yaml
  after: # default: []
    - flows/verify-upgrade.yaml

checks:
  crashes: true # default: true
  anr: true # default: true

output:
  format: console # console | json, default: console

CLI

upgradeguard test
  --config <path>     path to the configuration file (default: ./upgradeguard.yml)
  --from <apk>        APK of the version already installed (overrides the config)
  --to <apk>          APK of the version being upgraded to (overrides the config)
  --report <format>   console | json (default: console, or output.format)
  --verbose           also print the raw tool output behind the report
  --device <serial>   target this adb device serial

upgradeguard doctor
  Check that adb, an APK reader, Maestro and a device are available

Exit codes

| Code | Meaning | | ---- | ---------------------------------------------------------------------------------------------------------------------- | | 0 | Safe. Every step ran and no regression was found. | | 1 | A regression was found. The tool worked; the app did not. | | 2 | The upgrade could not be tested — configuration, environment, APK or tool error. Nothing was proven about the upgrade. |

Every thrown error maps to 2. A tool failure is never reported as a broken upgrade.

Example

This is the end-to-end validation run on 2026-09-02 against emulator-5554 (Pixel_10, API 36, x86_64) on Windows 11, with real adb, real apkanalyzer, real Maestro 2.10.0 and a real logcat dump. The full record is in docs/e2e-demo.md.

The bug

examples/demo-app persists onboarding state. Version 1.0 writes and reads onboardingCompleted. Version 1.1 writes and reads hasCompletedOnboarding. Version 1.1 is internally consistent, so a new user is fine. Only a user who already onboarded under 1.0 is sent back to onboarding, because their stored key has the old name. The fixed build adds a one-line migration that copies the old key forward.

Run 1 — upgrading to the buggy build: detected

$ upgradeguard test --config examples/demo-app/upgradeguard.yml

POST-UPGRADE

✗ Verification flow failed

Expected:
id: home-screen is visible


RUNTIME

✓ No native crash
✓ No ANR detected (best-effort — ANR log markers are unverified)

────────────────────────────

RESULT

❌ UNSAFE UPGRADE

1 regression detected

Exit code 1. The complete report is the one at the top of this README.

Run 2 — upgrading to the fixed build: passes

$ upgradeguard test --config examples/demo-app/upgradeguard.yml \
    --to examples/demo-app/builds/demo-1.1-fixed.apk

Identical report through RUNTIME, then:

POST-UPGRADE

✓ Verification flow passed

...

RESULT

✅ SAFE UPGRADE

No regressions detected

Exit code 0.

Run 3 — the control: the buggy build on a clean install passes

Run 1 on its own proves nothing. A tool that reported failure unconditionally would produce exactly the same first result, and would look just as successful at finding this bug. The control is what separates the two: the same buggy APK and the same verification flow, on a device that has never seen version 1.0.

===== CONTROL: CLEAN INSTALL of the BUGGY build (no v1.0 ever present) =====
Performing Streamed Install
Success
--- a brand new user logs in and onboards ---
 > Flow prepare-user
Launch app "dev.upgradeguard.demo"... COMPLETED
Tap on id: username-input... COMPLETED
Input text ada... COMPLETED
Tap on id: login-button... COMPLETED
Tap on id: finish-onboarding-button... COMPLETED
Assert that id: home-screen is visible... COMPLETED
Assert that "Welcome ada" is visible... COMPLETED
PREPARE EXIT=0
--- the SAME verification that failed on upgrade ---
Launch app "dev.upgradeguard.demo"... COMPLETED
Assert that id: home-screen is visible... COMPLETED
Assert that ".*Welcome.*" is visible... COMPLETED
Assert that "Finish onboarding" is not visible... COMPLETED
VERIFY EXIT=0

What the three runs establish

| Scenario | Result | | ------------------------------ | --------- | | Buggy build, clean install | passes | | Buggy build, upgraded from 1.0 | fails | | Fixed build, upgraded from 1.0 | passes |

The failure is caused by the upgrade path and by nothing else.

Two supporting measurements were captured the same day. The app's AsyncStorage SQLite database was hashed inside the app's private directory before and after adb install -r -t: identical digest, same uid, unchanged firstInstallTime, while versionCode advanced 1 → 2 — so data preservation is measured, not inferred. And Maestro's text matcher was confirmed to be case-insensitive, because Android uppercases the button label to FINISH ONBOARDING and a silently non-matching assertNotVisible would have made the control a false pass.

CI usage

UpgradeGuard needs a booted Android device, so it belongs in a job with an emulator, not in a plain ubuntu-latest step.

name: upgrade-guard
on: [workflow_dispatch]

jobs:
  upgrade-safety:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: 17
      - name: Install Maestro
        run: |
          curl -Ls "https://get.maestro.mobile.dev" | bash
          echo "$HOME/.maestro/bin" >> "$GITHUB_PATH"
      - uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 34
          target: google_apis
          script: npx upgradeguard test

Exit code 1 fails the job on a regression; exit code 2 fails it because the check could not run, which is a different problem and worth telling apart in your workflow.

Two caveats. The Maestro install one-liner above is Linux-only — on Windows runners Maestro is a manual zip install. And Maestro's behaviour on headless CI emulators has not been verified by this project; the only end-to-end runs so far are local, on a windowed emulator.

Limitations

• Android only. No iOS, no simulator support.
• One device at a time. No device matrix, no parallel runs.
• Verified on one configuration only: emulator-5554, x86_64, API 36, Play Store system image, Windows 11. The arm64 slice of an APK has never been run on physical hardware.
• ANR detection is best-effort. Android publishes no official log marker for an ANR, so the matcher is written against observed strings and has never fired against a real ANR. A miss means "no ANR detected", never "there was no ANR" — and the report says so. Crash detection is fully sourced and carries no such caveat.
• INSTALL_FAILED_* friendly messages cover only the codes actually observed. The AOSP strings are @hide and version-specific, so no closed mapping can be written. Unknown codes pass through verbatim, which is deliberate: a wrong entry cannot silently mis-report.
• Arguments containing cmd.exe metacharacters are refused, not executed. On Windows a .bat launcher is re-parsed by cmd.exe after Node has already quoted the command line, and no general-purpose escaping exists. A path or argument containing & | < > ^ % ! " is rejected with an error when the target tool is a Windows .bat.
• Maestro's exit code cannot be trusted, so classification depends on preflight checks removing every non-flow cause before a flow runs. This is structural, not a bug to be fixed.
• adb root is unavailable on Play Store system images, which blocks anything that would read the app's private data directory. v0.1 does not need it; use a google_apis image if you want it.
• The signer check is skipped when apksigner cannot be located. ADB then surfaces the signature mismatch instead, later and less clearly.
• Maestro's behaviour on headless CI emulators is unverified. See CI usage above.

Roadmap

Possible v0.2:

• GitHub Actions integration
• HTML report
• multiple post-upgrade flows
• better crash diagnostics
• automatic APK discovery
• more robust signing checks

Possible v0.3 — hostile-condition testing, reusing Maestro and Android capabilities rather than building a new E2E engine:

• offline upgrade validation
• slow network
• permission denied
• locale changes
• timezone changes
• low-storage conditions

Possible v1+, only if users ask for it:

• iOS simulator support
• AsyncStorage-aware assertions
• MMKV migration checks
• SQLite migration analysis
• release comparison history
• CI release gates
• HTML/dashboard reporting
• cloud device execution

Contributing

pnpm install
pnpm test        # vitest
pnpm lint        # eslint
pnpm typecheck   # tsc --noEmit
pnpm build       # tsc -p tsconfig.build.json

Every external process goes through src/infra/command-runner.ts, with argument arrays and never a shell. Paths use node:path. Tests come before implementation.

Bug reports are most useful with the output of upgradeguard doctor, your upgradeguard.yml, and the --verbose output of the failing run.

License

MIT — see LICENSE.