BeeOS Runner Agent

runner-agent is BeeOS's standalone local AI runtime for the BeeRunner Android accessibility app. It does not import, launch, or depend on device-agent.

Architecture

natural-language task
  -> OpenAI-compatible model
  -> observe / decide / act loop
  -> BeeRunner local Portal transport (preferred)
  -> ADB fallback when an action is known not to have executed
  -> Android device

Read operations such as screenshot and UI inspection can always retry through ADB. Mutating operations are only retried through ADB when BeeRunner explicitly rejected them before execution. A lost response after an action is treated as uncertain and is never replayed.

The runtime also holds an OS-level per-device lease. A second local Runner process fails with device_busy instead of racing the first process or overwriting its Portal authentication token.

This local mode does not require a BeeOS cloud bind. It requires USB debugging, BeeRunner installed, and accessibility enabled for full Unicode input and structure-aware control.

Install and build

pnpm install
pnpm build
pnpm link --global

Local controls

runner-agent doctor --serial bbdcbe5d
runner-agent screenshot --serial bbdcbe5d -o phone.png
runner-agent ui --serial bbdcbe5d
runner-agent apps --serial bbdcbe5d
runner-agent tap 500 800 --serial bbdcbe5d
runner-agent type "hello" --serial bbdcbe5d
runner-agent open-app com.tencent.mm --serial bbdcbe5d
runner-agent back --serial bbdcbe5d

Agent task

export BEEOS_RUNNER_BASE_URL="https://openrouter.ai/api/v1"
export BEEOS_RUNNER_API_KEY="..."
export BEEOS_RUNNER_MODEL="qwen/qwen3.5-plus"

runner-agent run "打开微信" \
  --serial bbdcbe5d \
  --observation-mode auto \
  --execution-mode fast \
  --output-dir .runner-traces/latest

The same settings can be passed as --base-url, --api-key, and --model. Secrets are used in memory and are never written to traces.

The initial release uses one provider-neutral JSON action contract (auto/general). Model-specific AutoGLM and QwenVL prompt/response adapters are intentionally deferred until their complete contracts can be ported and tested rather than approximated.

Observation modes:

• auto: screenshot plus UI structure; each source independently falls back to ADB.
• vision: screenshot only.
• xml: UI structure only.
• hybrid: screenshot plus UI structure.

Scope

The initial runtime intentionally contains no BeeOS account, binding, Message Service, ACP, A2A, cloud database, or Gateway code. Those are separate cloud integration layers.

Release

Every push and pull request verifies Node.js 20 and 22. A release tag sets the package version for the build, publishes npm, and creates a GitHub Release:

git tag runner-agent-v0.1.0
git push origin runner-agent-v0.1.0

The first release requires an NPM_TOKEN Actions secret with permission to create packages under @beeos-ai. After the package exists, configure npm Trusted Publishing for organization beeos-ai, repository beeos-runner-agent, and workflow release.yml; the workflow can then migrate to short-lived GitHub OIDC credentials. Prereleases use the next dist-tag; stable versions use latest.