ZelPi

A robotics toolkit that runs with one command — npx zelpi — plus a fleet-OS software twin for exploring the full Physical Intelligence architecture.

ZelPi gives a robotics developer, with zero install beyond Node:

• babyros — a built-in pub/sub middleware speaking the rosbridge v2 wire protocol. roslibjs, zelros, and existing rosbridge clients connect to it unchanged. No ROS installation required.
• SLAM + sensor fusion — occupancy-grid SLAM (correlative scan matching) and an EKF fusing odometry + IMU with online gyro-bias estimation, running as babyros nodes — or attached to your real robot's rosbridge.
• HAL — a versioned WebSocket protocol (e-stop, watchdog, skills, telemetry) linking the OS to any robot that runs the bundled zelpi-robot-agent, with a ROS 1/2 driver included.
• Real models — zelpi hub install downloads real VLA checkpoints (SmolVLA, ACT, Diffusion Policy) from HuggingFace and runs real inference; zelpi gpu setup switches the venv to CUDA.
• Honest MuJoCo simulations — a humanoid driven by a learned 40-DoF action model (zelpi sim), and a Shadow Dexterous Hand doing language-commanded pick-and-place (zelpi demo2).

Five minutes to a live SLAM map

npx zelpi slam launch        # simulated robot + lidar → EKF + SLAM → live ASCII map

That boots a babyros broker, a 2D differential-drive world with a 180-beam lidar (3% odometry slip, gyro bias — deliberately imperfect), an EKF fusion node, and an occupancy-grid SLAM node, then renders the map live in your terminal. On a reference run the fused estimate tracked the true pose at 0.05 m error vs 0.14 m from raw odometry, with the gyro bias estimated to within 3%.

Have a real robot publishing sensor_msgs/LaserScan + odometry on a rosbridge? Attach the same nodes to it:

npx zelpi slam attach --url ws://<robot>:9090 --scan /scan --odom /odom --imu /imu

More first commands:

npx zelpi babyros up                        # standalone middleware (port 9091)
npx zelpi babyros topics                    # zelros + roslibjs connect unchanged
npx zelpi hal connect ws://<robot>:9091     # native robot link (e-stop, skills, telemetry)
npx zelpi hub install smolvla               # real HuggingFace weights + local inference
npx zelpi gpu setup                         # CUDA torch for the models venv
npx zelpi docs hardware                     # the full real-hardware integration guide

Demo mode — the fleet-OS software twin

ZelPi also ships a complete simulated fleet OS: a cross-timescale neural stack (System 2 planning / System 1 skills / System 0 reflexes), an agent swarm, a Safety Broker with a Human-in-the-Loop gate, and a live fleet dashboard. It exists to make the architecture explorable — it is a deliberate simulation, not a claim that these commands drive production robots.

Every command backed by the software twin prints a ▓ SIMULATION banner so it can never be mistaken for real capability. Run zelpi demo for the exact split, or see the table below. (PIOS_NO_SIM_BANNER=1 suppresses the banner.)

npx zelpi intent "Inventory Aisle 4" plans with a real LLM call (bring your own key, or the zero-config hosted proxy) and now always prints plan source → LLM or plan source → deterministic parser, so you can tell exactly which System-2 path produced a plan.

What's real vs. simulated — read this before you rely on it

| Real & verified | Simulated / not yet verified end-to-end | |---|---| | babyros (lib/babyros/) — rosbridge v2 wire protocol, verified with zelros and roslibjs clients; SLAM/EKF verified live (0.05 m fused vs 0.14 m raw) | The fleet dashboard, System 2/1/0 tick loop, Safety Broker and agent swarm — the software twin behind the ▓ SIMULATION banner | | HAL protocol (cli/hal.mjs, server/robot-agent.mjs) — real WebSocket JSON wire format, e-stop/watchdog, 300+ unit tests | The "fine-tuning on fleet experience" step of hub pull — no real fleet-telemetry/RL training pipeline exists behind it | | ROS 1/2 bridge (server/createRosDriver.mjs) — verified against a real independent rclpy node, both directions | perceive / firewall (cli/spe.mjs) — deterministic, rule-based, no model call of any kind, by design | | zelpi intent — really calls an LLM (Google/OpenAI/Ollama or the hosted proxy) and labels every plan with its true source; falls back to a keyword parser, visibly, when the LLM is unreachable | Real hardware in the loop — every verification so far is sim-driver-to-sim-driver or sim-to-local-ROS-node; no physical robot has been tested yet | | hub install / hub pull — download real HuggingFace weights; vla_bridge.py runs genuine forward passes (SmolVLA, ACT, Diffusion Policy verified) | vla_bridge.py feeds synthetic camera observations — it proves the pipeline, not that a pretrained checkpoint is useful on your robot without fine-tuning | | GPU inference — verified on a real RTX 3060 (SmolVLA 56 s → 3.9 s/step) | One GPU model tested; NVIDIA/CUDA only — AMD/Apple accelerators unsupported | | Genie Sim install/deps (sim/scripts/geniesim_install.py, cli/geniesim.mjs) — verified end-to-end on real Linux (WSL2 Ubuntu 22.04) with real Docker + NVIDIA Container Toolkit + GPU passthrough all confirmed working; found and fixed 3 real bugs in the process (Windows symlink checkout, missing python3 fallback, an unsafe upstream interactive prompt) | Genie Sim's actual Isaac Sim runtime (geniesim up's Docker image build/docker up/ros2 launch, geniesim bridge's live ROS round-trip) — real code, reuses createRosDriver.mjs unmodified, but the Docker image itself was never built: it's a large, untested pull possibly gated behind an NVIDIA NGC login, and the one GPU tested against (RTX 3060, 6 GB) is below Isaac Sim's typical recommendation | | zelpi transporterdemo (Zelantrix floor-transfer robot) — the real, unmodified createRosDriver.mjs genuinely drives, docks, and e-stops a simulation of that robot's documented motor spec over a real babyros broker; computeLimits() derives velocity limits from motor spec + wheel/track geometry instead of a flat placeholder | The real robot itself — no physical unit was ever connected; its actual /cmd_vel//odom topic names, real safe speed limits, and any real hardware e-stop remain unconfirmed, see docs/HARDWARE.md | | Memory layer (cli/memory.mjs, lib/memory/polygres.mjs) — real semantic recall via Polygres (managed Postgres + pgvector) and Google gemini-embedding-001; verified live recalling a paraphrase sharing zero literal words with the original episode (81% cosine relevance) | Only exercised against one Polygres project so far; falls back to a local JSON file with keyword-overlap recall when DATABASE_URL isn't set — that path has no real semantic matching |

Full detail: docs/HARDWARE.md (including a Known Gaps section) and npx zelpi demo.

The CLI

npx zelpi                      # interactive OS shell — boots a backend automatically
npx zelpi help                 # full command surface

Highlights beyond the quickstart: zelros (ROS 1/2 CLI), zelpi hal scan (USB/serial hardware detection + model recommendation), zelpi hyworld / zelpi lingbot (world-model integrations), zelpi visualise (text → simulatable world), zelpi mcp (expose ZelPi to AI agents over MCP), and the conversational shell — type plain English at the REPL and it runs real commands to answer.

Point it at a full backend (live LLM / ROS / MLflow) with PIOS_HOST / PIOS_WS_URL. Full reference: docs/CLI.md and npx zelpi help. Protocol + driver-SDK docs for building on ZelPi: docs/PROTOCOL.md and docs/EXTENDING.md.

The web Observation Deck

A browser dashboard for the fleet software twin (the demo-mode half):

npm install
npm run dev      # → http://localhost:3000

Type an intent (Inventory Aisle 4, Retrieve the green box from Aisle 3) and watch the swarm decompose it, a robot drive the floor, and the Human-in-the-Loop gate pause a manipulation for your authorization.

• One engine, one rAF loop. PiOsEngine is framework-agnostic; React subscribes via useSyncExternalStore (lib/useEngine.ts).
• Deterministic world. Generated from a fixed seed (lib/world.ts) so reloads are reproducible.
• Stack: Next.js 15 · React 19 · TypeScript · hand-rolled engine · plain CSS.

Contributing & releasing

See CONTRIBUTING.md. Releases are tag-triggered (git tag v* → GitHub Actions tests → npm publish); the changelog is CHANGELOG.md. License: MIT.

Maintainers — build & publish the lean package locally:

npm run build:pkg            # → cli-dist/ (bundled engine + CLI, no web-app deps)
cd cli-dist && npm publish --access public