@kite3d/plugin-mujoco

MuJoCo WASM physics sessions and lightweight MuJoCo geometry views for Kite3D. This README is also the agent guide for using the package in a Kite3D project.

Renamed: this package was @blitzdev/plugin-mujoco until 0.1.1. Projects on the old name can run npx kite3d upgrade to rewrite the name in dependencies and kite3d.plugins.

The plugin hosts isolated simulations in module workers. Each session owns its model, controller or adapter, worker, and WASM heap. The built-in renderer can visualize common MuJoCo geometry or provide poses for authored meshes.

This package is a physics session host. It is not a Gymnasium task, training loop, reward function, observation wrapper, policy runtime, or automatic collider generator for arbitrary scene art. A custom adapter must supply environment-specific semantics.

Install and register

npm install @kite3d/plugin-mujoco

{"kite3d":{"plugins":["@kite3d/plugin-mujoco"]}}

The plugin name must also be an exact key in dependencies. npm install adds that key. Kite3D can then resolve the bare package name and add the exported plugin to the viewer. Find compatible packages with npm search keywords:kite3d-plugin.

This package requires the @kite3d/engine peer dependency at version 0.15.0 or newer.

Use

import {MujocoPhysicsPlugin} from '@kite3d/plugin-mujoco'

export async function main({viewer}) {
  const physics = viewer.getPlugin(MujocoPhysicsPlugin)
  const session = physics.createSession({
    modelUrl: new URL('./assets/robot/model.xml', import.meta.url).href,
    manifestUrl: new URL('./assets/robot/manifest.json', import.meta.url).href,
  })

  try {
    const description = await session.describe()
    const initial = await session.snapshot()
    const frame = await session.step(new Array(description.nu).fill(0), 5)
    await session.reset()

    const view = physics.createView(description)
    view.update(frame)
    viewer.scene.add(view.root)

    return {session, view, initial}
  } catch (error) {
    session.close()
    throw error
  }
}

Own the session and view for their entire lifecycle. In a component, retain the session immediately in start() and close it in stop(). Do not wait for loading to finish before retaining the handle. A RuntimeObjectOwner can own a Play-only view root. If code owns a view directly, call view.dispose().

API

createSession(config)

Creates and returns a session immediately. config.modelUrl is required for the generic simulator. An optional manifestUrl points to a JSON array containing the model XML filename and every relative asset filename that must be mounted with it. An optional runtimeUrl selects a different MuJoCo JavaScript loader.

The returned session exposes ready, onEvent(listener), and command(message) in addition to the methods below. Loading progress arrives through onEvent. One worker and one WASM heap are created per session.

load(config)

Creates a session, waits for session.ready, and returns it. It closes the session automatically when loading fails. Prefer createSession when Stop or another owner change must be able to cancel loading immediately.

describe()

Waits for loading and returns model dimensions, timestep, body names, actuator names and limits, supported geometry data, and the actual MuJoCo solver version.

step(action, frames = 1)

Applies exactly nu finite actuator values and advances 1 to 10,000 native physics ticks. The worker serializes requests. The result is a physics snapshot, not a Gymnasium observation, reward, termination, or truncation tuple.

reset({qpos?, qvel?, keyframe?} = {})

Resets all MuJoCo data, optionally selects a named or indexed keyframe, optionally applies finite qpos and qvel arrays, runs forward dynamics, and returns a snapshot. Environment randomization and custom numeric fields are adapter responsibilities.

snapshot()

Returns time, generalized positions and velocities, controls, sensor values, body positions and quaternions, geometry positions and rotation matrices, and contact count.

close()

Idempotently terminates the worker, releases its WASM heap, rejects pending calls, clears listeners, and removes the session from the plugin. Removing the plugin closes every remaining session. physics.sessionCount reports the current count.

createView(description)

Returns a ModelView. Add view.root to the appropriate authored preview or Play-only runtime root, call view.update(snapshot) for each displayed frame, and call view.dispose() when directly owned.

The renderer supports planes, spheres, capsules, ellipsoids, cylinders, boxes, and triangle meshes. It does not recreate textures, heightfields, flex or deformable geometry, or the complete MuJoCo renderer. Infinite planes use a bounded visual patch while their collision remains infinite. Unsupported geometry types throw.

adapterUrl

Pass an absolute browser URL as config.adapterUrl to replace the generic simulator methods with a task-specific worker adapter:

const session = physics.createSession({
  adapterUrl: new URL('./simulation/task-adapter.js', import.meta.url).href,
  modelUrl: new URL('./assets/robot/model.xml', import.meta.url).href,
})

The adapter module exports createAdapter({config, emit, loadModel}) and returns describe(), snapshot(), reset(args), and step({action, frames}). It may also return command(message) for non-blocking controls. Use emit(event) for progress or state notifications. The adapter owns observation ordering and normalization, action scaling, rewards, termination, policy state, and any environment-specific reset behavior.

WASM files and serving

The published package contains pinned, install-time-independent copies of runtime/mujoco.js and runtime/mujoco.wasm from @mujoco/mujoco 3.8.1. The default loader resolves both with import.meta.url, so keep the packaged directory structure intact. A server or bundler must expose the worker and runtime assets, serve JavaScript as JavaScript, and serve .wasm as application/wasm. Verify a production build as well as the Kite3D development server because asset URL rewriting differs between hosts.

If an MJCF references meshes or other files, list every file in the manifest beside the XML. Manifest paths must be relative, cannot contain empty or .. segments, and must include the XML entry itself.

Coordinates and memory ownership

MuJoCo is Z-up and Kite3D is Y-up. ModelView converts once by rotating its root by -pi / 2 around X. Do not repeat that conversion on child geometry. MuJoCo body quaternions use [w, x, y, z]; Three.js uses [x, y, z, w] when authored meshes are driven directly from body poses.

MuJoCo WASM vectors are live views into mutable heap memory. Never retain them as history or send them as durable state without copying. The generic snapshot() uses copies before the worker boundary. Adapter implementations must follow the same rule for every returned or emitted typed array.

Verify before claiming a port works

• Confirm the exact MuJoCo version, MJCF, included assets, initial free-root pose, home joint angles, actuator order, gain, bias, gear, control limits, torque limits, armature, friction, damping, solver settings, control frequency, and action latency.
• Compare deterministic resets and held-action trajectories against a trusted native MuJoCo run. Check dimensions and actual numeric states, not only whether the model loads or looks correct.
• Verify observation order and normalization, action scaling, recurrent or previous-action state, reward terms, termination, and truncation in the adapter. Importing MJCF alone does not reproduce a Gymnasium environment.
• Check that the Z-up to Y-up transform happens exactly once and that body quaternion ordering is converted correctly when driving authored meshes.
• Copy live WASM arrays before recording, replaying, emitting, or crossing asynchronous boundaries.
• For randomized or mutated geometry, compare actual bounds and contacts with a freshly compiled layout. Position agreement alone does not prove collision data is current.
• Test Play, Stop during loading, immediate Play again, reset, pause and resume, model load failure, and plugin removal. Confirm no workers, listeners, timers, runtime roots, or displaced materials survive cleanup.
• Verify supported primitives and triangle meshes in both development and production builds. Do not claim texture parity, heightfield support, deformable support, complete MuJoCo visual parity, WebGPU acceleration, or learned-policy performance unless each claim has its own evidence.

License and runtime provenance

The package is licensed under Apache-2.0. See LICENSE.

runtime/mujoco.js and runtime/mujoco.wasm are unmodified files from the Apache-2.0 package @mujoco/mujoco 3.8.1, repository directory wasm at google-deepmind/mujoco. The source projects pin the npm artifact with integrity sha512-uIIUcdAHG48N2UBmL/iLuBoAxH1xbTs/HaJthQd5euLvk44p9ivJ5wCc+IsnqgstpCdmCZAgx2l1WGwFyxErTQ==. The runtime's own Apache-2.0 license is retained at runtime/LICENSE.