Alex Coulombe Presents: XRSIM

Test VR apps without a headset. XR Simulator (xrsim) is an agent- and human-drivable harness that runs a real OpenXR application — Unreal, Godot, anything that speaks OpenXR — and lets you see it, move through it, and script it from an ordinary browser. It's the simctl + Simulator experience that iOS developers take for granted, brought to VR.

Presented by Alex Coulombe Presents · [email protected]

xrsim demo/xrsim help are free forever. Pointing it at a real app is not — a 14-day trial, then a license (free for active members, a one-time purchase otherwise). Full detail: Licensing.

Fastest possible start

You need Node.js 18 or newer (node --version to check) and nothing else — no headset, no VR app, no graphics card, no git clone.

npm install -g xrsim
xrsim demo

That's it. A simulated VR headset is now running and the cockpit opens in its own window — no tabs, no address bar, just the app (it's Chromium under the hood, launched in app mode; falls back to an ordinary browser tab if that's not available). It drives itself for a few seconds to show you it's live, then demonstrates catching a deliberately driven spawn-inside-geometry bug — the headset-free version of the real QA case study — before handing over: click the viewport and drive with W A S D (Q/E for down/up) — the built-in demo scene is a real, live render, so you'll see it move as you drive.

The viewport is a stereo pair — one view per eye, at a real 63mm IPD, the way a headset actually renders. Both controllers are drawn in the scene too, and they light up when a button fires — from the cockpit's own buttons, a gamepad, or a terminal:

xrsim input press --controller right --button trigger

Run that while watching the cockpit and the right controller flashes. That round trip — a command in one window, a real headset responding in another — is the whole product in one gesture.

That's a real render, not real video. demo runs a built-in test scene the harness draws itself; it's not the actual VR app's video (that needs a real CloudXR host — see below). Everything else is real too: as you move, the Pose readout on the right changes, and so does this:

xrsim state dump     # the headset's live position, as JSON
xrsim shutdown        # stop it

To see actual video you point xrsim at a real VR app — that needs a Windows machine with a GPU, and it's covered in the Quickstart.

Licensing: xrsim demo and xrsim help are free forever — no account, no trial, no limit. Pointing xrsim at a real app (xrsim launch, xrsim pico install/screenshot/logcat) starts a 14-day free trial the first time you run one of those, then needs a license — free for active Alex Coulombe Presents members, paid otherwise. See Licensing for exactly what's gated, how activation works, and why it verifies fully offline.

Two more things worth knowing up front:

• xrsim tutorial is a guided, five-step hands-on walkthrough, right in your terminal — start there if you just installed xrsim and aren't sure what to do next. xrsim help is the reference instead: every command in plain English, xrsim help <command> for one in full.
• Simulating a specific headset — xrsim demo --profile quest1 (or quest2, pico4, androidxr, …) throttles the demo scene to roughly that device's real frame rate instead of running flat-out.

Working on xrsim itself, or want a specific unpublished commit? Run from source instead:

git clone https://github.com/ibrews/xrsim.git
cd xrsim
npm install && npm run build
node bin/xrsim.js demo    # or: npm link, then plain `xrsim` everywhere — same as the docs below

See Distribution for the tarball-install alternative (no registry, no git) and what went into getting npm install -g xrsim working.

Setting up a whole room at once (a class, a workshop)? Don't make everyone type the two commands above by hand — one script installs Node.js (if it's missing) and then xrsim. Served via unpkg straight from the published npm package — not a GitHub link, since this repo's source is private (see Fastest possible start — that's normal, npm install -g xrsim doesn't need it). Recommended way to run it (skips a real macOS Gatekeeper gotcha — see below):

# macOS — paste into Terminal
curl -fsSL https://unpkg.com/xrsim@latest/install/install-mac.command | bash

# Windows — paste into PowerShell
irm https://unpkg.com/xrsim@latest/install/install-windows.bat -OutFile install-xrsim.bat; .\install-xrsim.bat

Prefer a file people can just double-click? Same scripts, downloadable directly: install-mac.command / install-windows.bat. On macOS specifically, a .command file downloaded from a browser gets Gatekeeper-quarantined, and double-clicking it does nothing (or shows "can't be opened" from an unidentified developer) until you right-click it and choose Open once. The curl | bash one-liner above skips that entirely, which is why it's the recommended path for a room full of people who've never seen this file before.

Want it to feel like a real Mac app, not a script? xrsim.app is a signed, notarized macOS app — double-click, no Gatekeeper warning, no Terminal required. It installs Node/xrsim on first launch (visibly, in a real Terminal window, so it doesn't look like nothing is happening) and opens the cockpit automatically once done; every launch after that just opens the cockpit directly. Source and rebuild instructions: install/macos-app. No Windows equivalent yet — that would need a paid, vetted code-signing certificate this project doesn't have set up.

Only gets you demo/help (no other setup needed for those) plus whatever a license unlocks — the pico and streamed/metaxrsim backends need their own installs; see Requirements.

Why it exists

Iterating on a VR experience normally means: build → put on the headset → look → take it off → change one thing → repeat. That loop is slow, it doesn't run in CI, and an AI agent can't do it at all. XR Simulator removes the headset from the loop:

• A live viewport in your browser shows exactly what the app is rendering.
• Standard movement — WASDQE, mouse-look, or a game controller — flies you through the scene.
• Everything is scriptable — pose, controller input, button presses, state assertions, screenshots, performance capture — as one-JSON-object-in, one-JSON-object-out commands an agent or a CI job can drive.

It has already been used to catch a real regression an artist could only have found in a headset: a game's "player spawns inside the terrain" bug, confirmed fixed entirely headset-free (see the QA case study).

How it works, in one picture

 your OpenXR app  ──OpenXR──▶  CloudXR runtime  ──▶  thin_host  ──WebRTC──▶  browser + IWER
 (UE / Godot / …)               (encodes frames)      (in-proc host)          (emulated Quest 3)
                                                                                   ▲
                                                        xrsim CLI / UI ────────────┘
                                                    (drives pose, input, asserts, streams video)

The app thinks it's talking to a real Quest. In reality its frames are streamed to a headless Chromium page running IWER (Meta's Immersive Web Emulation Runtime), which exposes a fully programmable emulated headset. xrsim drives that emulated device and mirrors the video back to you. Full detail: Architecture.

Start here

| I want to… | Read | |---|---| | Just see it work, right now | Fastest possible start above — one command | | Know exactly what has to be installed, per backend (streamed needs Windows+GPU+CloudXR, pico needs Android Studio+PICO's CLI, metaxrsim is macOS-only — none of that is needed for demo) | Requirements | | Set up the pico backend step by step, or check what's missing on this machine (xrsim setup opens a page that checks every prerequisite, tells you whether you're on the latest xrsim, and can launch the demo for you) | Setting up pico or run xrsim setup | | Package a VR app for PICO/Quest/Android XR and test it here — a class handout | Packaging & testing cheat sheet | | Know which JDK/NDK/Android SDK your Unreal version needs before packaging | Unreal → Android toolchain or run xrsim setup | | Get a scene running and drive it in 5 minutes | Quickstart | | See every command | Command reference | | Move around with WASDQE / a gamepad | Driving & input | | Use the browser cockpit | The UI cockpit | | Run several simulated headsets at once — one machine or a LAN, as a multiplayer test lab | Multi-instance & the fleet dashboard | | Understand the streaming pipeline | Architecture | | See it catch a real bug | QA case study | | Pitch this to another XR dev | The pitch | | Fix something that's not working | Troubleshooting | | Stop the launch-time "update available" notice | Set XRSIM_NO_UPDATE_CHECK=1 — see Staying up to date | | Know what's next | Roadmap | | Read the full development history (every bug, every fix, in detail) | Engineering notes |

Staying up to date

xrsim checks the npm registry once a day and, if you're behind, prints a one-line notice to stderr before running your command:

xrsim 0.7.0 is available (you have 0.6.4) — update with: npm install -g xrsim@latest

stdout stays exactly one JSON object, always — the notice never touches the machine-readable output contract, so scripts and agents are unaffected. The result is cached in ~/.xrsim/update-check.json, so only the first run in a 24-hour window touches the network (1.5s timeout, and it fails silently offline — a version nudge is never worth delaying real work over).

xrsim setup's page shows the same answer as a badge in its header — up to date, available, or couldn't check for updates when the registry was unreachable (never a silent "you're fine" it hasn't earned). It reads the same 24-hour cache, so opening the page costs no extra network round-trip.

Opt out entirely with XRSIM_NO_UPDATE_CHECK=1.

What's proven today

• Streamed backend, verified live on real hardware: stock Unreal (UE 5.7) and Godot 4.7 apps both stream, are drivable (pose + controllers + buttons), and produce real per-frame performance metrics — no headset, no per-app modification.
• One-command launch: xrsim demo for a simulated headset with zero configuration; xrsim launch --connect --app <exe> brings up the whole pipeline and the real app in the correct order.
• Live browser cockpit with WASDQE + gamepad driving and a real-time video viewport. The bundled demo scene renders a real stereo pair (per-eye, 63mm IPD) with both controllers drawn in-scene, lighting up on any button press — including one sent from a terminal, which makes the CLI↔cockpit round trip something you can watch rather than infer.
• Scriptable state assertions against a live app via the HarnessLink channel (Unreal).
• PICO backend, verified live: a real standalone Android/OpenXR APK (Unreal 5.8) runs fully immersive — real stereo scene, real reticle, real controller models — in PICO's own local GPU-accelerated Android emulator, driven entirely from the CLI (xrsim launch --backend pico --avd <name> --apk <path>).
• Quest-vs-PICO runtime detection: xrsim pico doctor <apk> tells you before you burn an install cycle whether an APK is packaged for Meta's OpenXR runtime — proven by patching a real Quest APK's manifest and watching PICO's own telemetry confirm the tag alone doesn't fix it.
• Real controller button injection on pico: xrsim input press dispatches a genuine Android KeyEvent into a running pico app — confirmed via logcat, including the app's own runtime firing its grab/interact handler in response. (Full head/controller pose injection is a documented, verified gap on this backend — see docs/pico-known-gaps.md.)
• Hand tracking, not just controllers: xrsim hand drives a 25-joint tracked hand — named poses, analog pinch, wrist placement, and a live joint read-back to assert on. The emulator ships three poses; xrsim hand register --pose fist --file fist.json adds your own (fist, open palm, peace, finger guns, per-finger pinches) and xrsim hand poses lists what's live. Verified against the real emulation runtime in a browser, including through frame.getJointPose(): a driven pinch closed the thumb-index gap the consuming app measures from 9.4 cm to 1.5 cm while the undriven hand stayed open as a control. The CloudXR.js client's upstream path is confirmed by source inspection too — it reads XRHand joints per frame, maps them to OpenXR's 26-joint layout (synthesizing the palm) and sends a handTrackingUpdate, with no flag to set; what remains unobserved is the Runtime→native-app hop, and the docs say exactly that rather than implying more. Note the client sends orientation only for the wrist and fingertips — position-only elsewhere, which matters if you drive a skinned hand rig from joint rotations. Use it to test gesture logic headlessly; real-world tracking (occlusion, frustum exits, motion blur) still needs hardware. See docs/driving-and-input.md.
• Android XR APKs, not just PICO-native ones: a real Android XR-targeted build (Galaxy XR-verified) renders its own declared scene on the PICO emulator — confirmed both visually and via PICO's own runtime telemetry tracking it as the focused, actively-rendering app.

Honesty is a feature here: every "verified" claim in these docs is backed by a command that was actually run against real hardware. Where something is designed but not yet proven end-to-end, the docs say so.

Things to Try

These assume xrsim is on your PATH — automatic with npm install -g xrsim, or run npm link from a source checkout. Building from source without linking? Write node bin/xrsim.js instead.

1. Get a headset running — xrsim demo. Your browser opens the cockpit; xrsim status reports active: true. (The viewport shows the built-in demo scene, not a real VR app — see Fastest possible start.)
2. Drive it and prove it moved — click the viewport and hold W for a second, then run xrsim state dump. The hmd_pose z value has changed. Driving from the CLI works too: xrsim pose move --hmd-delta "0,0,-1" --duration 1.0.
3. Run two headsets side by side — xrsim demo --name player1, then xrsim demo --name player2, then xrsim fleet ui --open. Two live tiles in one dashboard, each independently drivable. Full walkthrough (including a classroom LAN): Multi-instance & the fleet dashboard.
4. Watch an assertion fail honestly — xrsim script run with a state_query assert against an app that has no HarnessLink channel exits 1 with a clear reason, never a false pass. See Command reference.
5. Give the headset hands — xrsim hand mode --input hand, then xrsim hand pose --hand right --pose point, then xrsim hand joints --hand right. You get all 25 joints back; re-run the pose as pinch and watch index-finger-tip move. (hand pinch needs a connected app — without one it fails saying so, rather than pretending.)
6. Run the test suite — npm test. All of it runs against the bundled fixture, no GPU or headset required — including a contract suite that drives the real emulation runtime in a browser and asserts an app can see a pinch through frame.getJointPose().

Stuck on any of these? xrsim help lists everything, and Troubleshooting covers the failures worth knowing about.