droid-cua

AI-powered mobile testing desktop app for Android and iOS

Create, run, and manage mobile tests with natural language. The desktop app guides setup, connects to your target device or simulator, and turns AI exploration into reusable test scripts.

https://github.com/user-attachments/assets/b9e15a1d-8072-4a2f-a4c5-db180ae38620

droid-cua is a desktop app for AI-powered mobile testing.

It helps teams create, edit, and run tests for Android devices and emulators and iOS simulators on macOS using natural language instead of traditional test code.

Under the hood, droid-cua uses an AI agent to explore your app, execute actions, and save reusable test scripts that can also be run later in headless workflows.

1. Download the desktop app

• Mac: Download
• Mac (Intel): Download
• Windows: Download

2. Launch the app

Open the desktop app and choose the platform you want to test.

3. Choose your AI provider

In Settings, choose the CUA model you want to run. Loadmill - Smart uses your Loadmill sign-in and usage tracking. The OpenAI models use your own OpenAI API key.

4. Connect a target device

• Android: connect a device or select an emulator
• iOS: choose a simulator on macOS

5. Create or run a test

Use the desktop app to create a new test, edit an existing one, or run a saved script with live execution logs.

You can also keep project run history in a results folder and review past runs from desktop app reports.

• Android - Physical devices and emulators
• iOS - Simulators on macOS

Desktop app support:

• macOS: Android and iOS simulator workflows
• Windows: Android workflows only

Requirements

All platforms:

• Loadmill sign-in for Loadmill - Smart, or an OpenAI API key for the OpenAI models

Android:

• Android Debug Bridge (ADB)
• Android Emulator CLI for launchable emulators

iOS (macOS only):

• Xcode with iOS Simulator
• Appium
• XCUITest driver

• Desktop-first workflow - Create, run, and manage tests from one app
• Setup guidance - Configure API access and platform prerequisites in the app
• Device and simulator connection - Connect Android targets and iOS simulators
• Natural-language test creation - Describe flows in plain English
• Test management - Create, edit, save, and rerun reusable scripts
• Live execution logs - Watch actions and progress as tests run
• Reports and history - Review past runs from a project results folder inside the desktop app
• JUnit XML output - Write standard test reports for CI systems and external tooling
• Headless support - Reuse scripts in CLI and automation workflows

1. The desktop app connects to an Android device or emulator through ADB, or to an iOS simulator through Appium + XCUITest
2. Captures full-screen device screenshots
3. Scales down the screenshots for OpenAI model compatibility
4. Sends screenshots and user instructions to OpenAI's computer-use model
5. Receives structured actions such as click, scroll, type, keypress, wait, and drag
6. Rescales model outputs back to real device coordinates
7. Executes the actions on the device or simulator
8. Validates assertions and handles failures
9. Repeats until task completion

The desktop app is the primary way to use droid-cua.

For CI, scripting, or advanced workflows, droid-cua also includes a CLI for running saved instructions headlessly.

Desktop projects can also keep run reports in a results folder, including JUnit XML output that the app can read back as project history.

The recommended workflow is:

• design and debug tests in the desktop app,
• commit the .dcua file plus a headless CLI config file,
• run the same test headlessly in CI with --config for prompt parity.

Install:

npm install -g @loadmill/droid-cua

Examples:

# Interactive CLI
droid-cua

# Headless Android run
droid-cua --avd adb:emulator-5554 --instructions tests/login.dcua

# Headless Android folder run
droid-cua --avd adb:emulator-5554 --instructions tests

# Headless iOS simulator run
droid-cua --platform ios --avd "iPhone 16" --instructions tests/login.dcua

# Headless run with prompt-parity config
droid-cua --avd adb:emulator-5554 --instructions tests/login.dcua --config ci/droid-cua.json

# Headless LambdaTest real-device run
LAMBDATEST_USERNAME=... LAMBDATEST_ACCESS_KEY=... droid-cua \
  --device-source lambdatest \
  --platform android \
  --device-name "Galaxy S24" \
  --os-version "14" \
  --app ./app/build/outputs/apk/debug/app-debug.apk \
  --instructions tests/login.dcua \
  --config ci/droid-cua.json

# List AWS Device Farm remote-access devices
AWS_DEVICE_FARM_ACCESS_KEY_ID=... \
AWS_DEVICE_FARM_SECRET_ACCESS_KEY=... \
AWS_DEVICE_FARM_PROJECT_ARN=... \
droid-cua --device-source aws-device-farm --list-devices

# Headless AWS Device Farm real-device run
AWS_DEVICE_FARM_ACCESS_KEY_ID=... \
AWS_DEVICE_FARM_SECRET_ACCESS_KEY=... \
AWS_DEVICE_FARM_PROJECT_ARN=... \
droid-cua \
  --device-source aws-device-farm \
  --platform android \
  --device-name "Google Pixel 10 Pro" \
  --os-version "16" \
  --app ./app/build/outputs/apk/debug/app-debug.apk \
  --instructions tests/login.dcua

Example headless config:

{
  "cuaModel": "gpt-5.4",
  "promptCustomizations": {
    "basePromptInstructions": "",
    "designModeInstructions": "",
    "executionModeInstructions": ""
  },
  "appContextEnabled": true,
  "appContextPath": "../tests/context.md",
  "contextOptimizationEnabled": true,
  "contextOptimizationThreshold": 30000
}

Typical CI-style usage:

droid-cua \
  --avd adb:emulator-5554 \
  --instructions tests/login.dcua \
  --config ci/droid-cua.json \
  --debug

Supported CLI options include:

• --avd
• --platform
• --device-source
• --device-name
• --os-version
• --app
• --list-devices
• --browser
• --instructions
• --config
• --cua-model
• --context
• --no-context
• --context-optimization-threshold
• --no-context-optimization
• --base-prompt
• --execution-prompt
• --base-prompt-file
• --execution-prompt-file
• --secrets
• --record
• --report
• --debug

Config and precedence rules:

• Use --config <file> to supply prompt-affecting settings for headless runs.
• CLI flags override config file values.
• --context overrides the config app-context path.
• --no-context disables app context entirely.
• Context Optimization is enabled by default with a 30000 input-token threshold.
• --context-optimization-threshold <tokens> sets the threshold and enables Context Optimization.
• --no-context-optimization disables Context Optimization.
• --base-prompt and --execution-prompt let you pass prompt customizations inline on the command line.
• --base-prompt-file and --execution-prompt-file override the corresponding prompt customizations from config.
• If both inline and file-based prompt overrides are provided, the inline prompt flags win.

Secret typing:

• Put runtime typing secrets in a project .secrets file using dotenv-style KEY=value lines.
• Desktop execution and design mode load .secrets from the selected project folder. Headless CLI loads .secrets from the current working directory.
• Headless CLI also accepts --secrets KEY=value,OTHER=value; these values override .secrets for that run.
• The agent sees only secret keys and may call type_sensitive_text({ key }). Logs and reports may include keys, but not resolved values.
• Screenshot masking is out of scope. Secret values also necessarily pass through ADB, Appium, or browser automation while being typed.

HTML report (--report):

• Writes a self-contained HTML report of the run — the same Run Details view the desktop app shows, with screenshots inlined — that you can share and open in any browser.
• With no value it saves <test>--<date>--<time>.html (e.g. login--2026-06-03--16-09-20.html) in the current directory; pass --report <file> to pick a filename or --report <dir> to pick a folder.
• In the desktop app the same report is available from a run's Share → Download HTML report button.

droid-cua --avd adb:emulator-5554 --instructions tests/login.dcua --report ./reports

LambdaTest CLI runs:

• Use --device-source lambdatest with --platform, --device-name, --os-version, --app, and --instructions.
• Set LAMBDATEST_USERNAME and LAMBDATEST_ACCESS_KEY in the environment.
• Android LambdaTest runs require an .apk; iOS LambdaTest runs require an .ipa.
• The CLI uploads the app to LambdaTest for each run and closes the cloud session when the run finishes or fails.
• Each CLI LambdaTest run gets a unique LambdaTest build name so repeated local runs are easy to distinguish in the LambdaTest UI.

AWS Device Farm CLI runs:

• Use --device-source aws-device-farm with --platform, --device-name, --os-version, --app, and --instructions.
• Set AWS_DEVICE_FARM_ACCESS_KEY_ID, AWS_DEVICE_FARM_SECRET_ACCESS_KEY, and AWS_DEVICE_FARM_PROJECT_ARN in the environment (an IAM user with Device Farm access and the ARN of a Device Farm project in us-west-2).
• --device-source aws-device-farm --list-devices prints the remote-access device catalog (name, platform, OS) so you can pick valid --device-name/--os-version values, then exits.
• Android runs require an .apk; iOS runs require an .ipa. .app bundles are not supported.
• Re-running with the same app binary skips the upload: uploads are cached by checksum and reused for up to 30 days (Device Farm's upload retention).
• Device provisioning takes ~1 minute; the CLI shows live progress while it waits.
• The session is stopped when the run finishes, fails, or is interrupted with Ctrl-C — billing stops as soon as the stop is requested, so the CLI does not wait for the session to fully wind down.

AWS Device Farm in the desktop app:

• Open Integrations → AWS Device Farm and enter an AWS Access Key ID + Secret Access Key. The keys are validated on connect, a droid-cua Device Farm project is created automatically (the IAM user needs the minimal Device Farm policy, including ListProjects/CreateProject), and the remote-access device catalog loads. Credentials are stored in plaintext desktop-config.json, the same as BrowserStack/LambdaTest.
• On the Devices page, pick AWS Device Farm as the source, choose a device + an uploaded .apk/.ipa, and start a Design/Run session. Provisioning takes ~1 minute and live progress flows into the device log.
• Pause caveat: a Device Farm session paused for more than 5 minutes may be ended by AWS and need reconnecting.

Minimal GitHub Actions shape:

jobs:
  droid-cua:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm install -g @loadmill/droid-cua
      - run: |
          droid-cua \
            --device-source lambdatest \
            --platform android \
            --device-name "Galaxy S24" \
            --os-version "14" \
            --app ./app/build/outputs/apk/debug/app-debug.apk \
            --instructions tests/login.dcua \
            --config ci/droid-cua.json \
            --debug
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          LAMBDATEST_USERNAME: ${{ secrets.LAMBDATEST_USERNAME }}
          LAMBDATEST_ACCESS_KEY: ${{ secrets.LAMBDATEST_ACCESS_KEY }}

The same shape works for AWS Device Farm — swap the run step and env:

      - run: |
          droid-cua \
            --device-source aws-device-farm \
            --platform android \
            --device-name "Google Pixel 10 Pro" \
            --os-version "16" \
            --app ./app/build/outputs/apk/debug/app-debug.apk \
            --instructions tests/login.dcua \
            --config ci/droid-cua.json \
            --debug
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          AWS_DEVICE_FARM_ACCESS_KEY_ID: ${{ secrets.AWS_DEVICE_FARM_ACCESS_KEY_ID }}
          AWS_DEVICE_FARM_SECRET_ACCESS_KEY: ${{ secrets.AWS_DEVICE_FARM_SECRET_ACCESS_KEY }}
          AWS_DEVICE_FARM_PROJECT_ARN: ${{ secrets.AWS_DEVICE_FARM_PROJECT_ARN }}

Example without a config file:

droid-cua \
  --avd adb:emulator-5554 \
  --instructions tests/login.dcua \
  --context tests/context.md \
  --base-prompt "stop and look at the screen after every action you take."

Headless debug artifacts:

• --debug writes desktop-style structured JSONL artifacts under logs/.
• Each run creates logs/execution-<runId>-<timestamp>.jsonl.
• Each run also creates a sibling screenshot folder next to that JSONL file.
• Shared device events are written to logs/device-events.jsonl.
• --debug no longer creates the legacy logs/debug-*.log file for headless runs.
• If --debug and --record are both used, screenshots are written to both the debug artifacts folder and the legacy droid-cua-recording-<timestamp> folder.

Current headless behavior is documented in docs/headless-cli-spec.md.

© 2026 Loadmill. All rights reserved.