mujoco-react

Beta — This library is under active development. The API may change between minor versions until 10.0.

Composable React Three Fiber wrapper around the official @mujoco/mujoco WASM bindings. Load any MuJoCo model, step physics, render bodies, and write controllers as React components.

Demo | Docs | npm | Example Source | llms.txt

Install

npm install mujoco-react three @react-three/fiber @react-three/drei

Vite Plugin and Type-Safe Names

Use the Vite plugin to generate TanStack-style declaration merging for actuator, sensor, body, joint, site, geom, and keyframe names:

// vite.config.ts
import { defineConfig } from "vite";
import { mujocoReact } from "mujoco-react/vite";

export default defineConfig({
  plugins: [
    mujocoReact({
      models: {
        franka: "models/panda/scene.xml",
      },
    }),
  ],
});

The plugin writes src/mujoco-register.gen.ts during dev and build. Commit that generated file. Vite auto-loads it, so app code imports generated values from mujoco-react, not from the generated file:

// src/mujoco-register.gen.ts
// Auto-generated by mujoco-react. Do not edit.

import { registerRobotResources } from "mujoco-react";

const generatedRobotResources = {
  franka: {
    actuators: { joint1: "joint1", joint2: "joint2", joint3: "joint3", gripper: "gripper" },
    sensors: { force_sensor: "force_sensor", torque_sensor: "torque_sensor" },
    bodies: { link0: "link0", link1: "link1", hand: "hand" },
    joints: { joint1: "joint1", joint2: "joint2", joint3: "joint3" },
    sites: { tcp: "tcp" },
    geoms: { floor: "floor" },
    keyframes: { home: "home" },
  },
};

registerRobotResources(generatedRobotResources);

declare module "mujoco-react" {
  interface Register {
    robots: {
      franka: {
        actuators: "joint1" | "joint2" | "joint3" | "gripper";
        sensors: "force_sensor" | "torque_sensor";
        bodies: "link0" | "link1" | "hand";
        joints: "joint1" | "joint2" | "joint3";
        sites: "tcp";
        geoms: "floor";
        keyframes: "home";
      };
    };
    actuators: "joint1" | "joint2" | "joint3" | "gripper";
    sensors: "force_sensor" | "torque_sensor";
    bodies: "link0" | "link1" | "hand";
  }
}

Once generated, hooks like useCtrl, useSensor, useBodyState, and API methods like setCtrl, applyForce, getSensorData accept the global union. For robot-scoped code, use package exports such as RobotActuators.franka.gripper, RobotSites.franka.tcp, and RobotBodies.franka.hand. Generic type helpers like RobotActuators<"franka"> are still available for reusable library code. When no Register augmentation is present, names fall back to string.

Non-Vite projects can generate the same file with:

npx mujoco-react codegen franka=models/panda/scene.xml

Load a Model

import { MujocoProvider, MujocoCanvas } from "mujoco-react";
import type { SceneConfig } from "mujoco-react";

const panda: SceneConfig = {
  src: "https://raw.githubusercontent.com/google-deepmind/mujoco_menagerie/main/franka_emika_panda/",
  sceneFile: "scene.xml",
  homeJoints: [1.707, -1.754, 0.003, -2.702, 0.003, 0.951, 2.490],
};

export function App() {
  return (
    <MujocoProvider>
      <MujocoCanvas
        config={panda}
        camera={{ position: [2, -1.5, 2.5], up: [0, 0, 1], fov: 45 }}
        style={{ width: "100%", height: "100vh" }}
      />
    </MujocoProvider>
  );
}

Add Scene Tools

import { OrbitControls } from "@react-three/drei";
import { IkGizmo, RobotSites, useIkController } from "mujoco-react";

function PandaTools() {
  const ik = useIkController({ siteName: RobotSites.franka.tcp });

  return (
    <>
      <OrbitControls makeDefault />
      {ik && <IkGizmo controller={ik} />}
      <ambientLight intensity={0.7} />
      <directionalLight position={[1, 2, 5]} intensity={1.2} />
    </>
  );
}

Use it as a child of <MujocoCanvas>:

<MujocoCanvas config={panda}>
  <PandaTools />
</MujocoCanvas>

Write Controllers

import { useBeforePhysicsStep } from "mujoco-react";

function MyController() {
  useBeforePhysicsStep((_model, data) => {
    data.ctrl[0] = Math.sin(data.time);
  });

  return null;
}

Controllers are just React children that read sensors, write data.ctrl, apply forces, or call the MujocoSimAPI at physics-step time.

With generated resource values, reusable controllers can be scoped to one robot without hand-typing names:

import { RobotActuators, RobotJoints, RobotSites, useCtrl, useIkController } from "mujoco-react";

function FrankaTypedControls() {
  const gripper = useCtrl(RobotActuators.franka.gripper);
  const ik = useIkController({
    siteName: RobotSites.franka.tcp,
    joints: [
      RobotJoints.franka.joint1,
      RobotJoints.franka.joint2,
      RobotJoints.franka.joint3,
      RobotJoints.franka.joint4,
      RobotJoints.franka.joint5,
      RobotJoints.franka.joint6,
      RobotJoints.franka.joint7,
    ],
    actuators: [
      RobotActuators.franka.actuator1,
      RobotActuators.franka.actuator2,
      RobotActuators.franka.actuator3,
      RobotActuators.franka.actuator4,
      RobotActuators.franka.actuator5,
      RobotActuators.franka.actuator6,
      RobotActuators.franka.actuator7,
    ],
  });

  return null;
}

Use the Sim API

import { useMujoco } from "mujoco-react";

function ResetButton() {
  const sim = useMujoco();
  if (!sim.isReady) return null;

  return <button onClick={() => sim.api.reset()}>Reset</button>;
}

Map Controls to Joints

Use control groups when a robot's actuator order does not match a simple qpos[0..n] layout:

import { useRef } from "react";
import { resolveControlGroup, RobotSites, useBeforePhysicsStep } from "mujoco-react";
import type { ControlGroupInfo } from "mujoco-react";

function HoldTcpPose() {
  const armRef = useRef<ControlGroupInfo | null>(null);

  useBeforePhysicsStep((model, data) => {
    armRef.current ??= resolveControlGroup(model, { siteName: RobotSites.franka.tcp });
    if (!armRef.current) return;

    armRef.current.writeCtrl(data, armRef.current.readQpos(data));
  });

  return null;
}

resolveControlGroup() accepts { siteName }, { bodyName }, { joints }, or { actuators }. Selectors can be a name, ordered name array, regex, or predicate.

Build Observations

Build policy-ready observation vectors from common MuJoCo state without hard-coding offsets:

import { buildObservation, useBeforePhysicsStep } from "mujoco-react";

function PolicyDriver() {
  useBeforePhysicsStep((model, data) => {
    const obs = buildObservation(model, data, {
      qpos: true,
      qvel: true,
      ctrl: true,
      sensors: ["imu_gyro", "imu_accel"],
      sites: ["tcp"],
      projectedGravity: "torso",
    });

    runPolicy(obs.values, obs.layout);
  });

  return null;
}

Use output: "float64" when a downstream model expects double precision. Named resources are skipped when absent, so obs.layout is the source of truth for the current model.

WebSocket Control

Stream actuator commands over a WebSocket and send simulation state back. Use a schema validator such as Zod at this boundary because socket messages are untrusted app input (npm install zod for this example):

import { useEffect, useRef } from "react";
import { z } from "zod";
import { useMujoco, useBeforePhysicsStep, useAfterPhysicsStep } from "mujoco-react";

const CtrlCommand = z.preprocess((data) => {
  try {
    return typeof data === "string" ? JSON.parse(data) : data;
  } catch {
    return undefined;
  }
}, z.object({
  type: z.literal("ctrl_command"),
  ctrl: z.array(z.number()),
}));

function useWebSocketControls(url: string) {
  const wsRef = useRef<WebSocket | null>(null);
  const latestCtrlRef = useRef<number[] | null>(null);

  useEffect(() => {
    const ws = new WebSocket(url);
    wsRef.current = ws;

    ws.onmessage = (evt) => {
      const command = CtrlCommand.safeParse(evt.data);
      if (command.success) latestCtrlRef.current = command.data.ctrl;
    };

    return () => ws.close();
  }, [url]);

  // Apply incoming actuator controls each physics step.
  useBeforePhysicsStep((model, data) => {
    const ctrl = latestCtrlRef.current;
    if (!ctrl) return;
    for (let i = 0; i < Math.min(ctrl.length, model.nu); i++) {
      data.ctrl[i] = ctrl[i];
    }
  });

  // Send simulation feedback back after physics.
  useAfterPhysicsStep((model, data) => {
    const ws = wsRef.current;
    if (!ws || ws.readyState !== WebSocket.OPEN) return;

    ws.send(JSON.stringify({
      type: "feedback",
      time: data.time,
      qpos: Array.from(data.qpos),
      qvel: Array.from(data.qvel),
      sensordata: Array.from(data.sensordata),
    }));
  });
}

For reusable controllers with typed config, default merging, and children, use the createController factory:

import { createController, useCtrl, useBeforePhysicsStep } from "mujoco-react";

export const MyController = createController<{ gain: number }>(
  { name: "MyController", defaultConfig: { gain: 1.0 } },
  ({ config, children }) => {
    const shoulder = useCtrl("shoulder");

    useBeforePhysicsStep(() => {
      shoulder.write(config.gain * Math.sin(Date.now() / 1000));
    });

    return <>{children}</>;
  },
);

// <MyController config={{ gain: 2.0 }}>
//   <Debug showJoints />
// </MyController>

A createControllerHook factory is also available for the hook equivalent — see the Building Controllers guide.

Architecture

<MujocoCanvas> wraps R3F <Canvas> and forwards all Canvas props (camera, shadows, gl, etc.). For full control over the Canvas, use <MujocoPhysics> inside your own:

<MujocoProvider>                           <MujocoProvider>
  <MujocoCanvas config={...}>               <Canvas shadows gl={...}>
    <Scene />                                  <MujocoPhysics config={...}>
    <MyController />                             <MyController />
  </MujocoCanvas>                              </MujocoPhysics>
</MujocoProvider>                              <EffectComposer>...</EffectComposer>
                                             </Canvas>
                                           </MujocoProvider>

Custom IK Solvers

The built-in useIkController() uses Damped Least-Squares. Pass ikSolveFn to swap in your own solver (analytical, learned, etc.):

import { RobotSites } from "mujoco-react";
import type { IKSolveFn } from "mujoco-react";

const myIK: IKSolveFn = (pos, quat, currentQ) => {
  return myAnalyticalSolver(pos, currentQ); // return joint angles or null
};

const ik = useIkController({ siteName: RobotSites.franka.tcp, ikSolveFn: myIK });

useIkController(config | null)

Hook for interactive end-effector control. Pass null to disable IK (safe to call unconditionally):

import { IkGizmo, RobotSites, useIkController } from "mujoco-react";

const ik = useIkController({ siteName: RobotSites.franka.tcp });
return ik ? <IkGizmo controller={ik} /> : null;

| Config | Type | Default | Description | |--------|------|---------|-------------| | siteName | string | required | MuJoCo site to track | | joints | string \| string[] \| RegExp \| (joint) => boolean | inferred | Explicit hinge/slide joints for IK | | actuators | string \| string[] \| RegExp \| (actuator) => boolean | inferred | Explicit actuators for IK output | | numJoints | number | legacy only | Contiguous qpos/ctrl count from older examples | | ikSolveFn | IKSolveFn | built-in DLS | Custom solver function | | damping | number | 0.01 | DLS damping | | maxIterations | number | 50 | Max solver iterations |

Returns IkContextValue | null with methods like setIkEnabled, moveTarget, syncTargetToSite, solveIK, and getGizmoStats.

Pass the returned value to <IkGizmo controller={ik} /> or to your own controller as a prop.

By default the controller infers scalar hinge/slide joints by walking from the site body toward the model root. For robots where the MJCF control layout is not a simple chain, pass explicit names:

import { RobotActuators, RobotJoints, RobotSites } from "mujoco-react";

const leftArmIk = useIkController({
  siteName: RobotSites.franka.tcp,
  joints: [
    RobotJoints.franka.joint1,
    RobotJoints.franka.joint2,
    RobotJoints.franka.joint3,
    RobotJoints.franka.joint4,
    RobotJoints.franka.joint5,
    RobotJoints.franka.joint6,
    RobotJoints.franka.joint7,
  ],
});

const gripperIk = useIkController({
  siteName: RobotSites.franka.tcp,
  actuators: [
    RobotActuators.franka.actuator1,
    RobotActuators.franka.actuator2,
    RobotActuators.franka.actuator3,
    RobotActuators.franka.actuator4,
    RobotActuators.franka.actuator5,
    RobotActuators.franka.actuator6,
    RobotActuators.franka.actuator7,
  ],
});

Loading Models

The loader fetches src + sceneFile, parses the XML for dependencies (meshes, textures, includes), recursively fetches those too, and writes everything to MuJoCo's in-memory WASM filesystem.

// MuJoCo Menagerie
const franka: SceneConfig = {
  src: "https://raw.githubusercontent.com/google-deepmind/mujoco_menagerie/main/franka_emika_panda/",
  sceneFile: "scene.xml",
};

// Any URL
const custom: SceneConfig = {
  src: "http://localhost:3000/models/my_model/",
  sceneFile: "model.xml",
};

SceneConfig

interface SceneConfig {
  src: string;                      // Base URL for model files
  sceneFile: string;                // Entry XML/URDF file, e.g. "scene.xml"
  files?: File[];                   // Local files for browser upload workflows
  sceneObjects?: SceneObject[];     // Objects injected into scene XML at load time
  homeJoints?: number[];            // Initial joint positions
  xmlPatches?: XmlPatch[];          // Patches applied to XML files during loading
  onReset?: (model, data) => void;  // Called during reset after mj_resetData
}

Local Files and URDF

Load browser-selected MJCF or URDF files directly. Folder uploads preserve webkitRelativePath, and flat uploads fall back to matching referenced mesh/texture assets by basename:

import { useEffect, useRef } from "react";
import { useMujoco } from "mujoco-react";

function ModelUpload() {
  const sim = useMujoco();
  const inputRef = useRef<HTMLInputElement>(null);

  useEffect(() => {
    inputRef.current?.setAttribute("webkitdirectory", "");
  }, []);

  return (
    <input
      ref={inputRef}
      type="file"
      multiple
      onChange={(event) => {
        if (sim.isReady && event.currentTarget.files) {
          sim.api.loadFromFiles(event.currentTarget.files);
        }
      }}
    />
  );
}

Adding Objects to Any Scene

const config: SceneConfig = {
  src: "https://raw.githubusercontent.com/google-deepmind/mujoco_menagerie/main/franka_emika_panda/",
  sceneFile: "scene.xml",
  sceneObjects: [
    { name: "ball", type: "sphere", size: [0.03, 0.03, 0.03],
      position: [0.5, 0, 0.1], rgba: [1, 0, 0, 1], mass: 0.1, freejoint: true },
    { name: "platform", type: "box", size: [0.2, 0.2, 0.01],
      position: [0.4, 0.3, 0], rgba: [0.5, 0.5, 0.5, 1] },
  ],
};

XML Patching

xmlPatches: [{
  target: "panda.xml",
  replace: ["name=\"actuator8\"", "name=\"gripper\""],
  inject: "<site name=\"tcp\" pos=\"0 0 0.1\" size=\"0.01\"/>",
  injectAfter: "<body name=\"hand\"",
}]

Components

<MujocoProvider>

Loads the MuJoCo WASM module. Wrap your entire app in this.

| Prop | Type | Description | |------|------|-------------| | wasmUrl | string? | Custom WASM URL override | | mtWasmUrl | string? | Custom multi-threaded WASM URL override | | threadedLoader | (options?) => Promise<unknown> | Optional loader imported from @mujoco/mujoco/mt | | wasmVariant | "single" \| "threaded" \| "auto" | MuJoCo WASM build. Defaults to "single" | | timeout | number | WASM load timeout in ms | | onError | (error: Error) => void | Called if WASM fails to load |

The official @mujoco/mujoco package also ships a multi-threaded WASM build. Import it only in apps that opt into it:

import loadMujocoMt from "@mujoco/mujoco/mt";
import mtWasmUrl from "@mujoco/mujoco/mt/mujoco.wasm?url";

<MujocoProvider
  wasmVariant="auto"
  threadedLoader={loadMujocoMt}
  mtWasmUrl={mtWasmUrl}
>
  <App />
</MujocoProvider>

"auto" uses the threaded build only when threadedLoader and mtWasmUrl are provided and globalThis.crossOriginIsolated is true. Forced "threaded" mode requires threadedLoader, mtWasmUrl, Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp.

<MujocoCanvas>

Thin wrapper around R3F <Canvas>. Accepts all R3F Canvas props plus:

| Prop | Type | Description | |------|------|-------------| | config | SceneConfig | Required. Scene/robot configuration | | onReady | (api: MujocoSimAPI) => void | Fires when model is loaded | | onError | (error: Error) => void | Fires on scene load failure | | onStep | (time: number) => void | Called each physics step | | onSelection | (bodyId: number, name: string) => void | Called on double-click | | gravity | [number, number, number] | Override model gravity | | timestep | number | Override model.opt.timestep | | substeps | number | mj_step calls per frame | | paused | boolean | Declarative pause | | speed | number | Simulation speed multiplier |

<MujocoPhysics>

Physics provider for use inside your own R3F <Canvas>. Same physics props as <MujocoCanvas> without the Canvas wrapper. Accepts a ref for the MujocoSimAPI.

<MujocoProvider>
  <Canvas shadows camera={{ position: [2, 2, 2] }}>
    <MujocoPhysics ref={apiRef} config={config} paused={paused}>
      <MyController />
    </MujocoPhysics>
    <OrbitControls />
  </Canvas>
</MujocoProvider>

| Prop | Type | Description | |------|------|-------------| | config | SceneConfig | Required. Scene/robot configuration | | onReady | (api: MujocoSimAPI) => void | Fires when model is loaded | | onError | (error: Error) => void | Fires on scene load failure | | onStep | (time: number) => void | Called each physics step | | onSelection | (bodyId: number, name: string) => void | Called on double-click | | gravity | [number, number, number] | Override model gravity | | timestep | number | Override model.opt.timestep | | substeps | number | mj_step calls per frame | | paused | boolean | Declarative pause | | speed | number | Simulation speed multiplier |

<Body />

Declaratively add physics bodies to the simulation as JSX. Bodies are injected into the MJCF XML before model compilation.

<Body name="cube" type="box" size={[0.05, 0.05, 0.05]}
      position={[0.5, 0, 0.05]} rgba={[1, 0, 0, 1]}
      mass={0.1} freejoint />

// With custom Three.js visuals
<Body name="ball" type="sphere" size={[0.03, 0, 0]}
      position={[0, 0.3, 0.1]} mass={0.5} freejoint>
  <mesh>
    <sphereGeometry args={[0.03]} />
    <meshPhysicalMaterial color="gold" metalness={0.8} />
  </mesh>
</Body>

| Prop | Type | Default | Description | |------|------|---------|-------------| | name | string | required | Unique body name | | type | 'box' \| 'sphere' \| 'cylinder' | required | Geom type | | size | [number, number, number] | required | Geom size | | position | [number, number, number] | [0,0,0] | Initial position | | rgba | [number, number, number, number] | [0.5,0.5,0.5,1] | Color (ignored with children) | | mass | number? | -- | Body mass in kg | | freejoint | boolean? | -- | Add freejoint for free movement | | friction | string? | -- | MuJoCo friction params | | condim | number? | -- | Contact dimensionality (4-6 for grasping) | | children | ReactNode? | -- | Custom Three.js visuals |

<IkGizmo />

drei PivotControls gizmo that tracks a MuJoCo site and drives IK on drag. Requires a controller from useIkController().

| Prop | Type | Default | Description | |------|------|---------|-------------| | controller | IkContextValue | required | Controller from useIkController() | | siteName | string? | controller's site | MuJoCo site to track | | scale | number? | 0.18 | Gizmo handle scale | | onDrag | (pos, quat) => void | -- | Custom drag handler (disables auto-IK) |

<DragInteraction />

Click-drag to apply spring forces to bodies. Raycasts to find bodies, applies F = (mouseWorld - grabWorld) * body_mass * stiffness via mj_applyFT.

R3F Group Props

All visual components (DragInteraction, ContactMarkers, Debug, TendonRenderer, FlexRenderer) accept standard R3F group props like position, rotation, scale, visible.

<ContactMarkers visible={showContacts} />
<Debug showJoints scale={0.5} />

<ContactMarkers />

InstancedMesh showing MuJoCo contact points for debugging.

| Prop | Type | Default | Description | |------|------|---------|-------------| | maxContacts | number? | 100 | Max contacts to display | | radius | number? | 0.005 | Marker sphere radius | | color | string? | "#4f46e5" | Marker color | | visible | boolean? | true | Toggle visibility |

<SceneLights />

Auto-creates Three.js lights from MJCF <light> elements. Also available as useSceneLights(intensity?) hook.

<Debug />

Visualization overlays:

| Prop | Type | Default | Description | |------|------|---------|-------------| | showGeoms | boolean? | false | Wireframe collision geoms | | showSites | boolean? | false | Site markers | | showJoints | boolean? | false | Joint axes | | showContacts | boolean? | false | Contact force vectors | | showCOM | boolean? | false | Center of mass markers | | showInertia | boolean? | false | Inertia ellipsoids | | showTendons | boolean? | false | Tendon paths | | geomColor | string? | "#00ff00" | Color for wireframe geoms | | siteColor | string? | "#ff00ff" | Color for site markers | | contactColor | string? | "#ff4444" | Color for contact force arrows | | comColor | string? | "#ff0000" | Color for COM markers |

<TendonRenderer />

Renders tendons as tube geometry from wrap paths.

<FlexRenderer />

Renders deformable flex bodies from flexvert_xpos.

<InstancedGeomRenderer />

Opt-in renderer for repeated compatible geoms. It batches matching geom shape/material signatures into Three.js InstancedMesh objects and syncs instance transforms from data.geom_xpos / data.geom_xmat.

<ContactListener />

Component wrapper for contact events:

<ContactListener
  body="block_1"
  onContactEnter={(info) => console.log("contact!", info)}
  onContactExit={(info) => console.log("released", info)}
/>

<TrajectoryPlayer />

Plays back recorded qpos trajectories with scrubbing.

Hooks

useMujoco()

Access the simulation API (must be inside <MujocoCanvas> or <MujocoPhysics>). Narrow on isReady, isPending, or isError:

const sim = useMujoco();
if (sim.isReady) {
  sim.api.reset(); // fully typed
}

useMujocoWasm()

Access the raw WASM module lifecycle from any child of <MujocoProvider>. Most users won't need this — useMujoco() and hooks like useBeforePhysicsStep handle the model/data lifecycle for you.

import { useMujocoWasm } from "mujoco-react";

const { mujoco, status } = useMujocoWasm();

if (mujoco) {
  const model = mujoco.MjModel.from_xml_path("/path/to/scene.xml");
  const data = new mujoco.MjData(model);
  mujoco.mj_step(model, data);
  console.log(data.qpos);  // joint positions after one step
}

useBeforePhysicsStep(callback)

Run logic before mj_step each frame. Write to data.ctrl, apply forces, drive automation.

useBeforePhysicsStep((model, data) => {
  data.ctrl[0] = Math.sin(data.time);
});

useAfterPhysicsStep(callback)

Run logic after mj_step each frame. Read results, compute rewards, log telemetry.

useIkController(config | null)

Set up IK control for a MuJoCo site. Pass null to disable. Returns IkContextValue | null.

useCameraAnimation()

Standalone camera animation hook:

const { getCameraState, moveCameraTo } = useCameraAnimation();

// Animate camera over 1 second
await moveCameraTo(
  new THREE.Vector3(3, 0, 2),
  new THREE.Vector3(0, 0, 0.5),
  1000
);

useSensor(name) / useSensors()

Read sensor values by name. Returns a SensorHandle with read(), dim, and name:

import { RobotSensors, useSensor } from "mujoco-react";

const force = useSensor(RobotSensors.franka.force_sensor);
// force.read() -> Float64Array, force.dim -> number

useBodyState(name)

Position, quaternion, linear/angular velocity of a body (ref-based):

const { position, quaternion, linearVelocity, angularVelocity } = useBodyState("block_1");

useJointState(name)

Joint position and velocity:

const { position, velocity } = useJointState("joint1");

useCtrl(name)

Read/write actuator control by name. Returns a CtrlHandle with read(), write(), name, and range:

import { RobotActuators, useCtrl } from "mujoco-react";

const gripper = useCtrl(RobotActuators.franka.gripper);
// gripper.read() -> number, gripper.write(0.04), gripper.range -> [min, max]

useContacts(bodyName?) / useContactEvents(bodyName, handlers)

Query contacts or subscribe to enter/exit events:

useContactEvents("block_1", {
  onEnter: (info) => console.log("contact!", info),
  onExit: (info) => console.log("released", info),
});

useKeyboardTeleop(config)

Map keyboard keys to actuators:

import { RobotActuators, useKeyboardTeleop } from "mujoco-react";

useKeyboardTeleop({
  bindings: {
    "w": { actuator: "forward", delta: 0.1 },
    "s": { actuator: "forward", delta: -0.1 },
    "v": { actuator: RobotActuators.franka.gripper, toggle: [0, 0.04] },
  },
});

useGamepad(config)

Map gamepad axes/buttons to actuators:

useGamepad({
  axes: { 0: "joint1", 1: "joint2" },
  buttons: { 0: "gripper" },
  deadzone: 0.1,
});

usePolicy(config)

Framework-agnostic decimation loop for RL policies:

const obs = useObservation({ qpos: true, qvel: true, projectedGravity: "torso" });

const policy = usePolicy({
  frequency: 50,
  onObservation: () => obs.readValues(),
  onAction: (action, model, data) => applyAction(action, data),
});

buildObservation(model, data, config) / useObservation(config)

Build a flat Float32Array or Float64Array plus a layout map from qpos, qvel, ctrl, actuator activations, sensordata, named sensors, named site positions, and projected gravity.

useTrajectoryRecorder(config) / useTrajectoryPlayer(trajectory, config)

Record and play back simulation trajectories:

// Record
const recorder = useTrajectoryRecorder({ fields: ["qpos", "ctrl"] });
recorder.start();
// ... interact with simulation ...
recorder.stop();

// Play back recorded frames directly (no conversion needed)
const player = useTrajectoryPlayer(recorder.frames, {
  fps: 30,
  speed: 1.0,        // 0.5x, 1x, 2x, etc.
  loop: true,
  mode: "kinematic",  // or "physics" to replay ctrl through the sim
  onComplete: () => console.log("done"),
});
// player.play(), player.pause(), player.seek(42), player.setSpeed(2)
// player.state → "idle" | "playing" | "paused" | "completed"
// player.progress → 0-1

useVideoRecorder(config)

Record the canvas as video:

const video = useVideoRecorder({ fps: 30, mimeType: "video/webm" });
// video.start(), video.stop() -> returns Blob

useCtrlNoise(config)

Apply Gaussian noise to controls for robustness testing:

useCtrlNoise({ rate: 0.01, std: 0.05 });

useGravityCompensation(enabled?)

Applies qfrc_bias to qfrc_applied so joints hold position against gravity.

useActuators()

Returns actuator metadata for building control UIs.

useSitePosition(siteName)

Ref-based site position/quaternion tracking.

useBodyMeshes(bodyId)

Returns the Three.js meshes belonging to a MuJoCo body. Use for custom selection visuals, outlines, postprocessing, or any per-body mesh manipulation:

const meshes = useBodyMeshes(selectedBodyId);

// Use with drei Outline, or manipulate materials directly

useSelectionHighlight(bodyId, options?)

Convenience wrapper around useBodyMeshes that applies an emissive highlight:

useSelectionHighlight(selectedBodyId, { color: "#00ff00", emissiveIntensity: 0.5 });

useSceneLights(intensity?)

Hook form of <SceneLights>. Create Three.js lights from MJCF definitions imperatively:

useSceneLights(1.5);

MujocoSimAPI

The full API object available via ref or useMujoco() (when isReady):

Simulation Control

| Method | Description | |--------|-------------| | reset() | Reset sim, re-apply home joints | | setPaused(paused) | Set pause state | | togglePause() | Toggle pause, returns new state | | setSpeed(multiplier) | Set simulation speed | | step(n?) | Advance exactly n steps while paused | | getTime() | Current simulation time | | getTimestep() | Current timestep |

State Management

| Method | Description | |--------|-------------| | saveState() | Snapshot qpos, qvel, ctrl, time, act | | restoreState(snapshot) | Restore from snapshot | | setQpos(values) / getQpos() | Direct qpos access | | setQvel(values) / getQvel() | Direct qvel access | | setCtrl(nameOrValues, value?) | Set control by name or batch | | getCtrl() | Get all control values | | getControlMap() | Map directly actuated scalar joints to qpos/dof/ctrl addresses | | getActuatedJoints() | List scalar hinge/slide joints with matching actuators | | resolveControlGroup(selector) | Resolve qpos/ctrl mapping from a site, body, joint selector, or actuator selector | | buildObservation(model, data, config) | Build policy-ready observation vectors with layout metadata | | applyKeyframe(nameOrIndex) | Apply a keyframe | | getKeyframeNames() / getKeyframeCount() | Keyframe introspection |

Forces

| Method | Description | |--------|-------------| | applyForce(bodyName, force, point?) | Apply force via mj_applyFT | | applyTorque(bodyName, torque) | Apply torque via mj_applyFT | | setExternalForce(bodyName, force, torque) | Write to xfrc_applied | | applyGeneralizedForce(values) | Write to qfrc_applied |

Model Introspection

| Method | Description | |--------|-------------| | getBodies() | All bodies with id, name, mass, parentId | | getJoints() | All joints with id, name, type, range, bodyId | | getGeoms() | All geoms with id, name, type, size, bodyId | | getSites() | All sites with id, name, bodyId | | getActuators() | All actuators with id, name, range | | getSensors() | All sensors with id, name, type, dim | | getSensorData(name) | Read sensor value by name | | getContacts() | All active contacts | | getModelOption() | Timestep, gravity, integrator |

Model Mutation

| Method | Description | |--------|-------------| | setGravity(g) | Set gravity vector | | setTimestep(dt) | Set timestep | | addBody(body) | Add a SceneObject and recompile the scene | | removeBody(name) | Remove a generated SceneObject and recompile | | recompile(patches?) | Recompile current scene, optionally appending XML patches | | setBodyMass(name, mass) | Domain randomization | | setGeomFriction(name, friction) | Domain randomization | | setGeomSize(name, size) | Domain randomization |

Spatial Queries

| Method | Description | |--------|-------------| | raycast(origin, direction, maxDist?) | Physics raycast via mj_ray | | project2DTo3D(x, y, camPos, lookAt) | Screen-to-world raycast (returns bodyId + geomId) | | getCanvasSnapshot(w?, h?, mime?) | Base64 screenshot |

Scene Management

| Method | Description | |--------|-------------| | loadScene(newConfig) | Runtime model swap | | loadFromFiles(files, options?) | Load MJCF/URDF from a browser FileList |

Guides

Building Controllers

See Building Controllers for full patterns including config-driven controllers, IK gizmo coexistence, multi-arm support, and the createControllerHook/createController factories.

Contact Parameters

Objects that need stable contact (grasping, stacking, etc.) require tuned MuJoCo solver parameters — friction, solref, solimp, and condim. See Contact Parameters for details.

Click-to-Select

Combine R3F raycasting with useSelectionHighlight for body selection:

function ClickSelectOverlay() {
  const selectedBodyId = useClickSelect(); // your raycasting hook
  useSelectionHighlight(selectedBodyId);
  return null;
}

See Click-to-Select for the full implementation.

useFrame Priority

| Priority | Owner | Purpose | |----------|-------|---------| | -1 | MujocoSimProvider | beforeStep, mj_step, afterStep | | 0 (default) | SceneRenderer (internal), useIkController, your code | Body mesh sync, IK, rendering |

Roadmap

Features planned but not yet implemented:

| Feature | Priority | Description | |---------|----------|-------------| | Web Worker physics | P2 | Run mj_step off main thread via SharedArrayBuffer |

WASM Limitations (@mujoco/mujoco)

These MuJoCo features are not yet exposed in the WASM binding:

• flex_faceadr / flex_facenum / flex_face -- FlexRenderer renders vertices without face indices
• ten_rgba / ten_width -- TendonRenderer uses default color/width

License

Apache-2.0