@openbrt/audioctl

Generic WeClawBot audio/voice endpoint control CLI and AI-agent plugin.

It is meant for user-owned agents such as Codex, Claude Code, OpenClaw, Hermes, WorkBuddy, Gemini CLI, or any shell-capable agent runtime. It is not tied to one speaker brand. Any ESP32/Linux audio or voice endpoint that implements the WeClawBot MQTT/app envelope can use the same package. Music playback is one app on top of the endpoint; the same VM/capability model can later host TTS, voice remote control, smart-home control, and voice interaction apps.

Tell your agent

帮我在音箱建立一个工作 BGM 歌单，持续播放。请使用 @openbrt/audioctl 最新版。

Agent contract:

• If already bound, inspect the device and run the high-level task directly.
• If not bound, ask the user only for the short binding code, then bind and continue.
• Bluetooth/Wi-Fi provisioning is only for getting the device online and binding it to weclawbot.link. It must not decide whether the speaker is used for NetEase, Spotify, voice remote control, smart-home control, or anything else.
• Preferred BYOA flow: use a legal music platform API in the agent environment or another user-approved source to build a playable queue, then push it with audioctl queue QUEUE.json --play --json. The plugin suggests sources; the agent and user choose. NetEase Cloud Music Open Platform can be used this way to request the highest available quality URLs. Spotify should be driven through official Spotify Connect/Web Playback SDK or existing-user playback control; do not assume the Spotify Web API gives full-track direct audio URLs.
• If music-platform authorization is missing, use the current agent's official connector, OAuth flow, or secret store for that platform. Bluetooth/Wi-Fi provisioning is not a music-platform authorization entry and is not a product-purpose decision point.
• Never ask the user to paste AppSecret, PrivateKey, accessToken, refreshToken, Wi-Fi passwords, MQTT credentials, or other secrets into chat.

Install and bind

npm install -g @openbrt/audioctl
audioctl bind 123456 --name codex
audioctl doctor --online
audioctl inspect

One-shot usage:

npm exec --package @openbrt/audioctl -- audioctl bind 123456 --name codex

Credentials are saved in an audioctl-managed local config file. Agents should not inspect or parse that file; use audioctl bind, doctor, inspect, and the control commands instead. audioctl serializes concurrent commands that use the same local MQTT credentials, because the broker may require an exact MQTT client id. Agent-facing status is intentionally hardware-abstracted as audio_voice_ambient_endpoint; agents should not infer or search for a device brand/model/vendor from screenshots, BLE names, logs, or prior context. MQTT is a control plane and deployment path, not a realtime feedback data plane: agents should not read MQTT credentials, import mqtt, keep persistent MQTT connections, or run BPM/while/sleep loops that repeatedly publish LED/light commands. Continuous effects must be started as a device-local light timeline, VM app, or compatibility feedback effect, then left to run on the device.

Commands

audioctl play
audioctl pause
audioctl toggle
audioctl next
audioctl previous
audioctl volume 48
audioctl volumeup
audioctl volumedown
audioctl status
audioctl inspect --json
audioctl light describe --json
audioctl light set all --color amber --brightness 0.35
audioctl light timeline ./hero.light.json
audioctl light bind ./hero.light-bindings.json
audioctl light clear
audioctl wakeword status --json
audioctl wakeword configure ./wakeword.json --json
audioctl wakeword bind ./wakeword-bindings.json --json
audioctl wakeword trigger work_mode --source agent --json
audioctl firmware check --json
audioctl firmware update --yes --json
audioctl report "center LED active=true but user cannot see it" --kind capability_mismatch --area light --include-inspect --json
audioctl feedback signal ready
audioctl feedback flash cyan --repeat 2
audioctl feedback volume 66
audioctl feedback beep
audioctl feedback clear
audioctl queue ./work-bgm.queue.json --play
audioctl workbgm --play
audioctl rule ./my-playlist-rule.json
audioctl deepnight
audioctl prompt 123456 --agent codex --alias 视听房音箱

inspect asks the device for a redacted configuration/status snapshot over the existing MQTT channel. Agent scripts can read Wi-Fi connection state, BLE provisioning state, wake-word capability, playback, VM/app status, firmware runtime version, and health without receiving Wi-Fi passwords, MQTT credentials, or music-platform secrets.

wakeword is the logical voice-trigger binding layer. It does not train or replace the low-level acoustic wake model. It maps existing device wake events, ASR phrases, or agent-injected test phrases to VM/app events or safe local actions. Agents should read audioctl inspect --json first and use reported wakeword.commands / voice.wakeword.* capabilities, not product-specific vendor research.

audioctl wakeword status --json
audioctl wakeword configure ./wakeword.json --json
audioctl wakeword bind ./wakeword-bindings.json --json
audioctl wakeword trigger work_mode --source agent --json
audioctl wakeword clear --json

Example wakeword config:

{
  "schema": "weclawbot.wakeword.config.v1",
  "mode": "logical_voice_trigger",
  "language": "zh",
  "phrases": [
    { "id": "work_mode", "text": "开始工作", "aliases": ["工作模式"] }
  ],
  "bindings": [
    {
      "phrase_id": "work_mode",
      "action": "app.event",
      "feedback": ["feedback.beep", "feedback.signal:wake"]
    }
  ]
}

firmware check reads the official firmware index from https://weclawbot.link/firmware/audio/manifest.json, compares it with inspect.firmware.version, and reports whether a host/runtime update is available. firmware update is intentionally gated: without --yes it only prints the update plan and confirmation_required. An agent must show the version, source and notes to the user first, then run audioctl firmware update --yes --json only after the user explicitly confirms. Firmware updates are for the device host/runtime and VM manager; they do not erase Wi-Fi, MQTT binding, music-platform settings, queues, rules, or installed VM app slots.

audioctl firmware check --json
audioctl firmware update --json        # dry plan, no mutation
audioctl firmware update --yes --json  # runs only after explicit user approval

light controls expose programmable visual surfaces. Agents should use audioctl light describe --json to inspect surfaces such as all, ring, and center, plus event sources and runtime limits. For non-trivial visuals, build a weclawbot.light.timeline.v1 file and optionally a weclawbot.light.bind.v1 file so the device can start/stop visuals on local events such as media.playing, media.paused, and media.ended.

audioctl light describe --json
audioctl light timeline ./hero.light.json
audioctl light bind ./hero.light-bindings.json
audioctl light clear

Example timeline:

{
  "schema": "weclawbot.light.timeline.v1",
  "id": "hero_ambient",
  "loop": true,
  "stop_on": ["media.paused", "media.ended", "media.error"],
  "duration_ms": 3750,
  "tracks": [
    {
      "surface": "ring",
      "keyframes": [
        { "t": 0, "color": "#ff6000", "brightness": 0.18 },
        { "t": 470, "color": "#ff6000", "brightness": 0.55 },
        { "t": 940, "color": "#301800", "brightness": 0.12 }
      ]
    },
    {
      "surface": "center",
      "keyframes": [
        { "t": 0, "color": "#ffd080", "brightness": 0.08 },
        { "t": 1875, "color": "#ffd080", "brightness": 0.40 },
        { "t": 3750, "color": "#301800", "brightness": 0.05 }
      ]
    }
  ]
}

feedback/led controls expose simple semantic device feedback. Agents should prefer semantic signals (ready, busy, success, error, playing, paused, volume_step_up, volume_step_down, button_press) so each hardware adapter can map them to its native LED ring, small screen, haptic motor, or speaker prompt. Explicit LED controls are also available:

audioctl feedback signal playing
audioctl feedback flash '#00ffff' --repeat 2
audioctl feedback volume 42
audioctl feedback beep
audioctl feedback clear
audioctl led flash amber --repeat 1

feedback effect is a compatibility/demo shortcut for old agents. New agent work should prefer light describe + light timeline + light bind, because named effects such as music_beat, pulse, or breathing do not describe the actual hardware topology. feedback effect requires an explicit effect name; calling it with no name is rejected.

Physical keys/touch surfaces are reported as VM capabilities such as input.touch.read, input.gesture.read, and input.button.bind. A VM app can declare those capabilities and bind local gestures to safe device actions. The agent should derive the binding from the user's goal and the reported capabilities, instead of assuming a fixed product-specific layout. These bindings are device-local for low latency and do not require the agent to be online for every tap.

queue --play is the generic agent-built music path. The agent uses the user's authorized music provider outside the speaker, writes a weclawbot.playable_queue.v1/schemaVersion: 1 queue containing only safe media URLs and redacted metadata, then pushes it to the device. The queue must not contain platform private keys, OAuth tokens, cookies, custom request headers, or Wi-Fi/MQTT credentials. Expiring media URLs are allowed; refreshing them is the agent/app's job.

workbgm --play installs a persistent work-bgm-autopilot VM app on the audio device, writes the bundled 工作 BGM playlist rule through the device-side music adapter, requests a refresh, and starts playback once. The app expresses a continuous BGM intent and respects later user-initiated pause/resume controls. If the device reports netease_auth_refresh_failed, that only means the legacy device-side NetEase runtime cannot refresh itself. The better agent behavior is to switch to the BYOA queue/app path: obtain the user's music-platform identity in the agent environment, build a fresh queue, and push it with audioctl queue. Only if the user explicitly wants the legacy device-side NetEase runtime should the agent say a separate device-side music-account settings entry is required. That entry must be separate from Bluetooth/Wi-Fi provisioning.

deepnight pushes the bundled “深夜工作” playlist rule and asks the device to refresh/play according to its local runtime. The package never contains music platform private keys.

Control envelope

The MQTT control message is:

{
  "schema": "weclawbot.control.v1",
  "id": "audio_<uuid>",
  "kind": "music_command",
  "music": {
    "command": "play"
  }
}

Supported commands include play, pause, toggle, next, previous, seek, set_volume, volume_up, volume_down, status, and set_rule.

Feedback messages use the same MQTT envelope:

{
  "schema": "weclawbot.control.v1",
  "id": "feedback_<uuid>",
  "kind": "feedback_command",
  "feedback": {
    "command": "flash",
    "color": { "red": 0, "green": 255, "blue": 255, "alpha": 255 },
    "repeat": 2
  }
}

Supported feedback commands are signal, flash, volume, effect, beep, and clear. Devices that do not report a matching feedback.* capability should reject the command instead of pretending to support it.

Light messages use a distinct command envelope:

{
  "schema": "weclawbot.control.v1",
  "id": "light_<uuid>",
  "kind": "light_command",
  "light": {
    "command": "timeline",
    "timeline": {
      "schema": "weclawbot.light.timeline.v1",
      "id": "custom_visual",
      "loop": true,
      "stop_on": ["media.paused", "media.ended"],
      "duration_ms": 2000,
      "tracks": [
        {
          "surface": "all",
          "keyframes": [
            { "t": 0, "color": "#202020", "brightness": 0.05 },
            { "t": 1000, "color": "#ff6000", "brightness": 0.5 }
          ]
        }
      ]
    }
  }
}

Supported light commands are describe, set, timeline, bind, and clear. Continuous light timing and event reactions run on the device VM.

Agent feedback reports

audioctl report is the agent-to-maintainer feedback channel for issues, suggestions, capability mismatches, UX gaps, and documentation gaps. It writes a redacted JSON report to ~/WeClawBot/agent-feedback by default, or to WEC_AUDIO_REPORTS_DIR / --reports-dir DIR when set.

User observation wins over status. A device returning applied, active=true, or a capability entry only means the runtime accepted the command; it does not prove the user can physically see, hear, or feel the result. If a user reports that a center LED is not visible, a touch gesture is unreliable, or a beep is not audible, agents should fall back to a confirmed visible/audible/controllable behavior, then create a local report:

audioctl report "center LED surface is reported active but not visually observable by the user" \
  --kind capability_mismatch \
  --area light \
  --observed "light_active=true, user still cannot see center LED" \
  --expected "visible center-light cue or capability should be marked unavailable" \
  --include-inspect \
  --json

Reports are local by default. Agents must ask the user before posting a report to a public issue tracker, chat, email, or any other external system. Reports must not include MQTT credentials, music-platform tokens, Wi-Fi passwords, or other secrets.

Wakeword messages configure the logical voice-trigger layer:

{
  "schema": "weclawbot.control.v1",
  "id": "wakeword_<uuid>",
  "kind": "wakeword_command",
  "wakeword": {
    "command": "configure",
    "config": {
      "schema": "weclawbot.wakeword.config.v1",
      "mode": "logical_voice_trigger",
      "phrases": [
        { "id": "work_mode", "text": "开始工作" }
      ],
      "bindings": [
        { "phrase_id": "work_mode", "action": "app.event" }
      ]
    }
  }
}

Supported wakeword commands are status, configure, bind, trigger, and clear. This is a VM/app event binding layer, not low-level acoustic-model training.

Firmware update messages also use the same MQTT envelope, but carry only an official release descriptor. The device downloads the artifact itself from weclawbot.link, enforces HTTPS, checks SHA-256 and byte size, installs from a staging directory, restarts local services, and reports health through inspect.

{
  "schema": "weclawbot.control.v1",
  "id": "firmware_<uuid>",
  "kind": "firmware_update",
  "firmware": {
    "schema": "weclawbot.firmware.update.v1",
    "confirmed_by_user": true,
    "release": {
      "schema": "weclawbot.firmware.release.v1",
      "device_class": "audio_voice_ambient_endpoint",
      "runtime": "rokid_music_runtime",
      "version": "0.1.2",
      "channel": "stable",
      "artifact": {
        "url": "https://weclawbot.link/firmware/audio/rokid-music-runtime-0.1.2.tar",
        "sha256": "<64 hex chars>",
        "bytes": 123456
      }
    }
  }
}