@h2loop/uav — Hydron UAV Domain Pack

ArduPilot/PX4 flight-log intelligence for Hydron: decode a .bin, run deterministic system checks, persist the observation locally, and compare against the airframe's prior flight. Built as an external Hydron plugin (no core rebuild).

→ USAGE.md — install, validate, and everyday use (start here). Also: CHANGELOG.md · PACKAGING.md (distribution/IP).

Design docs: internal-docs/wiki/engineering/features/domain-plugins/uav/.

Status

This cycle = P1 + P1b + P2 (verified against the real sample log 17-03-2026 Sortie 4.bin):

• ✅ Decode — pure-TS FMT-driven DataFlash decoder, no runtime deps, validated field-by-field against the 73 oracle CSVs (src/decode/). TimeUS kept as bigint.
• ✅ Decode dual-stack — TS .bin; pymavlink .tlog/.log; PX4 .ulog (pyulog). Same DecodedLog → the same checks run on PX4.
• ✅ Check — 23 system checks + param audit over in-memory bun:sqlite: power (cell-V, sag, current, internal-resistance, brownout), control (attitude, rate-tracking, hover, rcou/rcin, thrust-margin), sensors (vibration + alignment/degradation classifier, imu, mag/offsets, gps, EKF/EKF2 velocity+position innovations, variance), ESC, and ERR/MODE event findings. Deterministic; missing tables → SKIP.
• ✅ Param audit + grounding — safety/range/consistency over PARM, ranges grounded in ArduPilot's apm.pdef.xml metadata (3419 params bundled; KG path for online).
• ✅ Tuning (USP) — uav_tune: FFT → dominant peak → recommended INS_HNTCH_* notch params.
• ✅ MAVLink param link — bridge Tier-C param_get/set (gated --allow-param-write) for the push + retest loop; flight C2 intentionally absent.
• ✅ Persist + run-tracking — local episode store + keyed diff; uav_tag run labels + airframe-profile baseline (diffAgainstProfile).
• ✅ PlotJuggler handoff — uav_export writes one CSV per message type.
• ✅ Control plane — on-demand uav skill + references, the uav-investigate / uav-compare / uav-verify subagents (spawned automatically), /uav-* commands, built-in threshold / param-rule defaults (overridable per project via .hydron/uav-thresholds.yaml / param-rules.yaml), bundled apm-params.json. No dedicated UAV mode — the domain rides in the normal modes.

Layout

src/
  types.ts              # LOCKED shared contracts (DecodedLog, Finding, Report, EpisodeDocument, Thresholds)
  index.ts              # plugin entry — registers uav_analyze, uav_query, uav_param_audit
  decode/               # FMT-driven .bin decoder (TS port of split_dataflash_log.py) + tests
  harness/
    analyze.ts          # the per-flight pipeline orchestrator
    loadtables.ts       # DecodedLog -> in-memory SQLite
    checks.ts           # the 21 system checks + param audit -> Finding[]
    thresholds.ts       # uav-thresholds.yaml loader (+ built-in v1 defaults)
    extract.ts          # DecodedLog -> entity/episode/params/events/measurements
  store/
    store.ts            # episode JSON + index.jsonl (system of record)
    diff.ts             # keyed param/finding diff
script/analyze.ts       # standalone smoke harness (no hydron runtime needed)
agents/                 # subagents (uav-investigate / uav-compare / uav-verify)
commands/               # /uav-* slash commands
skills/uav/             # on-demand `uav` skill + references
data/                   # bundled apm-params.json + built-in defaults
bridge/                 # optional MAVLink→MCP Python bridge
plugins/                # BUILT bundle (index.js + worker/) — generated
mcp.json                # exa MCP server (disabled by default); mavlink is registered
                        #   from src/index.ts's config hook — it needs a resolved
                        #   interpreter + an ABSOLUTE script path, which static JSON can't give

Run standalone

bun install
bun run script/analyze.ts "/path/to/flight.bin" --project /path/to/workdir
bun test          # 29 tests: decoder vs oracle, checks, store/diff
bun run typecheck

Knowledge grounding (the retrieval ladder)

Verdicts are always deterministic (harness vs bound). Sourcing a bound's value + citation walks a ladder, first hit wins, never invents:

1. uav-thresholds.yaml — tuned / customer-stated bounds (local, authoritative).
2. query_hydron_project — ingested OEM datasheets in the project KG/RAG (cite citation).
3. Exa.ai web MCP (exa_web_search_exa / exa_web_fetch_exa) — datasheets / rules-of-thumb not in the KG; cite the URL. Pluggable: swap for another search MCP. These tools are available in the normal modes; the UAV subagents allow-list them so they don't prompt per call.

Verified sample result (17-03-2026 Sortie 4.bin)

ArduCopter V4.6.3, ~48 min, 1306 params. battery CRITICAL (2× ERR 6/1 failsafes), vibration HIGH (VibeZ 40.4 > 30), hover HIGH (avg ThH 0.17 < 0.25 band); attitude/ altitude/rcou/rcin/mag/ekf clean. Decode ~0.4 s.

RAM / large-file scaling (P0 — design-partner blocker)

AirBound requires 1.5–2 GB files and ~4-flight compares; we'd only tested to ~60 MB, and big files froze the machine. This is the top priority. See internal-docs/.../uav/11-airbound-scale-and-deep-integration.md.

✅ Landed (2026-06-29) — P-scale steps 1–5 (streaming decode → on-disk SQLite). analyzeFlight/uav_query now stream .bin rows straight into a throwaway on-disk SQLite (harness/db.ts openAnalysisDb/createSqliteSink; decode/dataflash.ts decodeDataflashInto; harness/load.ts decodeToDb), retaining only the low-rate tables extract.ts/vehicle.ts read in JS (RETAIN_TABLES = PARM/MSG/GPS/ERR/MODE); high-rate aggregates moved to SQL in extract.ts. The 24 checks were already SQL-only. Verified by src/harness/streaming.test.ts (synthetic .bin parity/concurrency/cleanup + a real-bin smoke). Synthetic benchmark (115 MB / 5M-row log): peak RSS 781 MB → 282 MB (~2.8×) — the eliminated part is the JS row graph (the term that scales linearly and freezes at GB scale).

✅ P1b landed — chunked file read. decode/dataflash.ts decodeCore now reads the file in 4 MiB windows and carries cross-boundary messages, instead of arrayBuffer()-ing the whole file. Peak RSS is now independent of file size.

Real benchmark (the 1.69 GB golden 2026-04-07 15-20-25.bin): streaming analyze completes at ~332 MB peak RSS in ~79 s (down from ~1.08 GB pre-P1b; the old full path would be 10 GB+ and freeze). Retained JS rows tiny (PARM 1397 / MSG 117 / ERR 2). AirBound's 1.5 GB gate is met, RSS flat in file size. loadTables (:memory:) and decodeAuto are unchanged for uav_tune/export/plot.

Test hygiene: the golden flight log lives outside the repo and changes; src/test-fixture.ts centralizes it. Full-decode / pymavlink / CSV-oracle tests skipIf(!goldenFullDecodeSafe()) (skip when the golden is absent or >500 MB — they'd OOM on a GB log); the memory-safe streaming path is exercised on the real golden via hasGolden(). Suite: 172 pass / 16 skip / 0 fail.

CPU: off-thread decode + duty-cycle throttle — ✅ landed

The streaming decode trades memory for CPU (millions of batched inserts + parse of tens of millions of messages on one thread). Two cross-platform controls (no nice/OS-priority):

• Off the main thread (#3): analyze/uav_query run the decode in a Worker (harness/worker-entry.ts + worker-client.ts), so Hydron's main loop stays responsive and the OS schedules the work on another core. The full on-disk SQLite is still built inside the worker — later uav_query/root-cause keep full row specificity. If a Worker can't run, it falls back to in-thread (the feature never breaks).
• CPU ceiling (#4): set UAV_DECODE_MAX_CPU (e.g. 0.6 ≈ 60% of a core) to duty-cycle the decode (harness/throttle.ts, applied per-chunk in decodeCore). Unset = full speed. Trades wall-clock for a CPU cap — cross-platform (timer-based). Verified in src/harness/throttle-worker.test.ts (strict worker path matches in-thread; throttle idles).

Multi-flight compare at GB scale (P-multicompare) — ✅ landed

uav_compare compares 2+ already-analyzed flights at once — finding-severity trend, key- metric trend (vibe_peak, hover_thr, …), and parameter drift across the runs. It diffs the small stored per-flight EpisodeDocuments (≈100 KB each), never the raw logs, so comparing four 1 GB flights costs ~four 100 KB docs, not 4 GB (analyze each first — each streams at ~flat RAM). New: store/diff.ts diffEpisodesN (N-way), store/store.ts readEpisodeById, harness/compare.ts compareFlights, and the uav_compare tool. Tested in src/harness/compare.test.ts.

Root cause (in this code): uav_analyze holds three concurrent copies of the log — the whole file buffer, every row of every message type as boxed JS arrays in log.tables (decode/dataflash.ts), and a full copy again in an :memory: SQLite (harness/loadtables.ts). The JS row graph dominates and OOMs on big logs. The 24 checks already run as pure SQL aggregates over the DB, so the fix is structural:

• P0: loadtables.ts → on-disk throwaway SQLite (not :memory:) + journal_mode=OFF / synchronous=OFF / bounded cache_size. (Partial — removes one copy.)
• P1 (the real fix): stream decode rows directly into the on-disk DB in batches; retain in JS only the low-rate tables extract.ts/vehicle.ts iterate (PARM/MSG/MODE/ERR/GPS); move extract.ts's duration/vibe_peak/hover aggregates to SQL. High-rate tables (IMU/RATE/ATT/VIBE/BAT/CTUN) then live only on disk; RAM is bounded by cache_size.
• P1b: read the file in 1 MB chunks (reuse the contentHash sliding-buffer) instead of arrayBuffer(); P2: size-based "large-file mode", per-table row caps with partial flags, optional child-process decode so a blowup can't freeze the host.
• Multi-file: the streaming/on-disk design must hold for 4 concurrent flights (uav-compare), each its own on-disk DB, diffed via SQL joins.

Known limitations / next

• Multidisciplinary root cause (P1, AirBound litmus test) — beyond the current metric checks, rank + eliminate competing hypotheses (aero × firmware × electronics) and cite each; first demo target: improperly tuned pitch-gain filter (RATE oscillation × VIBE/FFT × ATC_RAT_PIT_*/INS_GYRO_FILTER params).
• Pluggable diverse ingest (P1) — weather-station CSV, custom params, external sensor logs, time-aligned with the flight (self-registering handler layer).
• Deep UAV-tool integration (the moat) — SITL replay to confirm a hypothesis, deeper pymavlink/MAVProxy + param-metadata + firmware code-context grounding.
• Correlation/trend tier (oscillation FFT, compass-motor interference, EKF drift over time) and param airframe-profile diff (auditAgainstProfile stub) are not yet built.
• PX4 .ulog isn't wired yet; the pymavlink path is tested on the ArduPilot .bin.
• Live uav_watch/uav_stream and the uav-diagnose orchestrator (core-gated) are future.
• uav-thresholds.yaml overlay parsing is minimal (built-in v1 defaults are authoritative until the YAML parse is fleshed out).

Install (plugin system)

The pack is now a proper Hydron plugin bundle. Install it with the hydron plugin CLI:

# Global (available in every project on this machine):
hydron plugin add @h2loop/uav

# Project-scoped (committed to repo, shared with the team):
hydron plugin add @h2loop/uav --project

hydron plugin add validates the bundle, then COPIES it into the plugins root (global → ~/.config/hydron-cli/plugins/uav; --project → <repo>/.hydron/plugins/uav). A local-path install (hydron plugin add /path/to/uav-pack [--project]) is also supported for dev. Once installed, agents, commands, skills, tools, MCP servers, and the ambient domain context are all discovered and loaded automatically — no manual config wiring needed.

Scope & persistence. The install persists across sessions — install once, it's picked up at every future startup. Global = "this machine does UAV work" (available in every repo); --project = "this repo is UAV work" (committed with the repo, shared with the team). A project's local uav-thresholds.yaml still overrides the global one.

Uninstall with hydron plugin remove uav (add --project for the project-scoped install). Other commands: hydron plugin list (installed plugins + status), hydron plugin enable <name> / disable <name>, hydron plugin update <name>, hydron plugin validate <path> (static analysis before installing).

After install or uninstall, fully quit and relaunch Hydron — the plugin loads once per process (a window reload isn't enough).

UAV is a domain LAYER, not a mode. Every mode (Ask / Architect / Code / Debug) becomes UAV-aware via a short always-on signpost (the experimental.chat.system.transform hook) plus the on-demand uav skill. There is no dedicated UAV agent or mode; the uav-investigate, uav-compare, and uav-verify subagents handle isolated multi-step loops and are spawned automatically, not selected by the user.

MCP prerequisites

The plugin ships two MCP servers (both disabled by default). Enable them per-project in your Hydron config:

Exa (web grounding)

Provides exa_web_search_exa / exa_web_fetch_exa for OEM datasheet limits not in the project KG.

Requires: EXA_API_KEY environment variable (get one at https://exa.ai).

MAVLink bridge (live link)

A local Python stdio server. Read-only tools: read_params, param_get, param_get_all, get_messages, tail_telemetry, get_flight_mode, get_status. The write tier (param_set / param_set_batch) is on by default so tune→push→retest works out of the box — see below.

Requires:

• Python ≥ 3.10 with pymavlink>=2.4.49 and mcp[cli]>=1.0.0,<2 installed. The pack does not auto-provision a venv — install the deps yourself: pip install -r bridge/requirements.txt (ideally into a venv), and point the command's python3 at that interpreter.
• A reachable MAVLink endpoint (e.g. udp:127.0.0.1:14550 from SITL or a telemetry radio), or a .tlog/.log file for offline replay.
• The bridge/ directory ships with the plugin (the mcp.json references it).

Ships enabled, with param writes on — the command passes --no-read-only --allow-param-write, so param_set / param_set_batch are registered and the tuning loop works with no configuration. ⚠ this lets the agent write params to a live vehicle — prefer SITL/bench. To ship read-only, drop both flags.

No --connect is passed, so the target comes from the MAVLINK_CONNECT env var, falling back to udp:127.0.0.1:14550:

export MAVLINK_CONNECT=serial:/dev/ttyUSB0:57600      # COM3:57600 on Windows

Don't put it in the server's environment block — Hydron spreads that over process.env, so it would shadow your export instead of defaulting to it.

Connect-target forms: udp:HOST:PORT (default udp:127.0.0.1:14550 = SITL/MAVProxy) | tcp:HOST:PORT | serial:/dev/ttyUSB0:BAUD | /path/log.tlog (offline replay); can also be set via the MAVLINK_CONNECT env on the server. "Connected" ≠ a live vehicle: the panel shows Connected once the bridge process starts, but tools only return data while something is actually publishing MAVLink at the connect target; otherwise calls time out (-32001). Plain .bin log analysis via uav_analyze does not need the bridge.

Known limitations

1. Loader collision (host bug) — FIXED. Hydron's plugin dedup used to key on the hook module's basename (index), so two plugins each shipping a plugins/index.ts hook could collide and one would be silently dropped. This is now resolved in hydron-cli: getPluginName keys on the plugin install directory, not the basename, so multiple hook-bearing plugins coexist.

2. Local-path install copies everything: hydron plugin add ./path does a raw fs.cp with no files/.npmignore filtering. If you ran bun install or bun run build locally, the copy includes node_modules + built artifacts + source. Publish to npm and install via spec (hydron plugin add @h2loop/uav) for a clean install — npm respects files.