babylon-vrm

VRM porting to babylon.js.

Refactored with AI assistance from virtual-cast/babylon-vrm-loader, maintained at cnb.cool/opensource.cool/vtuber/babylon-vrm. 0.1.0 is the first npm release of the refactor.

What's new in 0.1.2

• Performance: zero-allocation per-frame update paths (LookAt / expressions / node constraints / spring bones / animation retargeting) — steady-state avatar updates no longer create GC pressure; the node-constraint topological sort is cached after load instead of re-running every frame; idle frames (no active expression) skip the morph/material update pass; MToon reuses a scratch vector for its per-bind _Time uniform and no longer depends on window.performance (non-browser runtimes safe).
• New utility: optimizeMorphTargets() pins the morph shader variant on Babylon.js 9 texture-backed morph targets (numMaxInfluencers), stopping vertex-shader recompiles whenever expressions activate or deactivate (e.g. blinking).
• Memory: VRM1Manager.dispose() releases the parsed glTF nodes JSON; the VRM 0.x collider pass hoists the world-to-center matrix out of the per-collider loop and reads absoluteScaling components without intermediate arrays.

What's new in 0.1.1

• Package hardening: top-level main entry for tooling that does not read exports, richer npm metadata (keywords, author, bugs, homepage), PBRMaterial imported via a deep @babylonjs/core path instead of the root barrel, and an explicit ESM-only note.
• README fixes: the VRM 1.0 sample now imports the VRM1Manager type; assets that never shipped in the tarball were dropped.
• Attribution & license: state that this package is an AI-assisted refactor of virtual-cast/babylon-vrm-loader, preserve the original MIT copyright line and add the current maintainer.

What's new in 0.1.0

• Fix VRM expressions having no visual effect: MToonMaterialDefines was missing MORPHTARGETS_POSITION and the MORPHTARGETTEXTURE_HAS* layout defines, so position morphing was compiled out of the MToon vertex shader even though expression weights were applied correctly. The missing defines are now declared, matching StandardMaterialDefines.
• Fix MToon shader compilation: register the core shader includes referenced by MToon, and drop the RECIPROCAL_PI2 macro clashing with helperFunctions.
• Fix .vrma animation: glTF loader auto-play of VRM 1.0 AnimationGroups is disabled; playback is driven by VRM1AnimationController.update() only.
• Build: declaration files are emitted by tsc (build:dts) instead of rolldown-plugin-dts, which OOM'd while bundling @babylonjs types; npm run build now completes reliably. A build:tsc pipeline emits standard-decorator JavaScript (dist-tsc) for direct-browser debugging.

Requirements

• babylon.js ^9.23.0 (@babylonjs/core and @babylonjs/loaders)
• an ESM-capable bundler or runtime — the package is ESM-only ("type": "module", no CommonJS build)

Features

• Supports .vrm v0.x file loading
  • with extensions.VRM glTF Extension
• Supports .vrm v1.0 (VRM 1.0) file loading
  • with VRMC_vrm 1.0 extension
    • Humanoid (with required bone validation and legacy thumb bone name compatibility)
    • Expressions (preset/custom, additive accumulation, overrideBlink / overrideLookAt / overrideMouth, material color / UV transform binds)
    • LookAt (bone / expression appliers with linear RangeMap, head animation compensation)
    • FirstPerson mesh annotations (auto headless model, per-node annotations)
  • with VRMC_springBone 1.0 extension
    • sphere / capsule colliders, dependency-sorted joint update, center space
    • VRMC_springBone_extended_collider 1.0: inside sphere/capsule + plane colliders
  • with VRMC_node_constraint 1.0 extension
    • roll / aim / rotation constraints
  • with VRMC_materials_mtoon 1.0 extension
    • mapped onto the vendored MToonMaterial (with UniVRM shading parameter migration)
    • shadingShiftTexture and matcapFactor supported natively
  • with VRMC_materials_hdr_emissiveMultiplier extension (archived spec)
• Supports .vrma v1.0 (VRMC_vrm_animation) animation files (GLB container)
  • humanoid retargeting (T-pose normalization, hips leg-length scaling)
  • expression and lookAt tracks, loop playback
  • usage: load the .vrma (its own scene), then scene.metadata.vrm1Animations[0].createController(vrm1Manager)
  • per frame: controller.update(deltaTime) then vrm1Manager.update(deltaTime)
• VRMUtils equivalents
  • combineMorphs (per-expression morph merge), removeUnnecessaryVertices, deepDispose
• Supports .vci file loading
• Supports MToonMaterial (vendored from babylon-mtoon-material)
• Get bone(TransformNode) from Unity Humanoid bone mapping name
• BlendShape morphing
  • with preset name constants(VRMBlendShapePresetName) and weight getter
• SpringBone
• LookAt eye tracking
  • both Bone and BlendShape lookAt types
• FirstPerson mesh annotations
  • assigns layerMask per annotation and creates headless model for Auto meshes
• Meta properties(license fields included)
• Supports VCI features
  • VCAST_vci_material_unity
  • VCAST_vci_meta (license fields included)
  • VCAST_vci_embedded_script (Lua source decoded from the GLB bufferView)
  • VCAST_vci_audios (audio clips decoded from the GLB bufferView)
  • VCAST_vci_item (SubItem grabbable / scalable settings per node)
  • VCAST_vci_collider (box / sphere / capsule / mesh colliders per node)
  • VCAST_vci_rigidbody (rigidbody settings per node)
  • VCAST_vci_joints (fixed / hinge / spring joints per node)
  • physics data is exposed as definitions bound to each TransformNode, ready to be wired into a Babylon.js physics engine

Usage

with npm

$ npm install --save @babylonjs/core @babylonjs/loaders babylon-vrm

import * as BABYLON from '@babylonjs/core';

// has side-effect
// registers the VRM/VCI file loader and glTF extensions
import 'babylon-vrm';

// vrmFile is File object retrieved by <input type="file">.
const scene = await BABYLON.SceneLoader.LoadAsync('file:', vrmFile, engine);

// VRM 0.x: manager is in scene.metadata.vrmManagers
const vrmManager = scene.metadata.vrmManagers[0];
// VRM 1.0: manager is in scene.metadata.vrm1Managers
const vrm1Manager = scene.metadata.vrm1Managers[0];

// Update secondary animation
scene.onBeforeRenderObservable.add(() => {
    const deltaTime = scene.getEngine().getDeltaTime();
    vrmManager?.update(deltaTime); // VRM 0.x
    vrm1Manager?.update(deltaTime); // VRM 1.0
});

// Model Transformation
vrmManager.rootMesh.translate(new BABYLON.Vector3(1, 0, 0), 1);

// Work with HumanoidBone
vrmManager.humanoidBone.leftUpperArm.addRotation(0, 1, 0);

// Work with BlendShape(MorphTarget)
vrmManager.morphing('Joy', 1.0);

// LookAt(eye tracking)
vrmManager.lookAt.target = scene.activeCamera; // or any TransformNode / BABYLON.Node

// FirstPerson view
// assigns layerMask to meshes and creates headless model for `Auto` meshes
vrmManager.firstPerson.setup();
camera.layerMask |= vrmManager.firstPerson.firstPersonOnlyLayerMask;

// Meta properties
console.log(vrmManager.meta.title, vrmManager.meta.licenseName);

VRM 1.0

import type { VRM1Manager } from 'babylon-vrm';

// VRM 1.0 manager is registered in scene.metadata.vrm1Managers
const manager = scene.metadata.vrm1Managers[0] as VRM1Manager;

// Humanoid bones (hip/leftEye/... 55 bones, camelCase)
manager.humanoid.head.addRotation(0, 1, 0);

// Expressions
// weight is applied on manager.update(deltaTime)
manager.expressionManager.setValue('happy', 1.0);
console.log(manager.expressionManager.getValue('happy'));

// LookAt
manager.lookAt.target = scene.activeCamera;

// FirstPerson (same layerMask API as 0.x)
manager.firstPerson.setup();
camera.layerMask |= manager.firstPerson.firstPersonOnlyLayerMask;

VRM Animation (.vrma)

// .vrma is a glTF asset loaded into its own scene (GLB container)
const animScene = await BABYLON.SceneLoader.LoadAsync('file:', vrmaFile, engine);
const vrmAnimation = animScene.metadata.vrm1Animations[0];

const modelScene = await BABYLON.SceneLoader.LoadAsync('file:', vrmFile, engine);
const vrm1Manager = modelScene.metadata.vrm1Managers[0];
const controller = vrmAnimation.createController(vrm1Manager);

scene.onBeforeRenderObservable.add(() => {
    const deltaTime = scene.getEngine().getDeltaTime();
    controller.update(deltaTime); // retargets bones / expressions / lookAt
    vrm1Manager.update(deltaTime); // then apply expressions -> constraints -> spring bones
});
```

### VCI

```ts
// VCI manager is registered in scene.metadata.vciManagers
const vciManager = scene.metadata.vciManagers[0];

// Meta properties (license fields included)
console.log(vciManager.meta.title, vciManager.meta.author, vciManager.meta.modelDataLicenseType);

// Embedded script (Lua source text, decoded from the GLB bufferView)
const script = vciManager.embeddedScript.scripts[vciManager.embeddedScript.entryPoint];
console.log(script.name, script.text);

// Audio clips (raw bytes, decoded from the GLB bufferView)
const clip = vciManager.audios[0];
const sound = new BABYLON.Sound(clip.name, clip.data, scene);

// SubItems (VCAST_vci_item): grabbable / scalable settings per node
for (const subItem of vciManager.subItems) {
    console.log(subItem.node.name, subItem.item.grabbable, subItem.item.groupId);
}

// Physics definitions (VCAST_vci_collider / rigidbody / joints) are bound to nodes
for (const entry of vciManager.colliderNodes) {
    console.log(entry.node.name, entry.colliders[0].type); // 'box' | 'sphere' | ...
}
for (const entry of vciManager.rigidbodyNodes) {
    console.log(entry.node.name, entry.rigidbodies[0].mass);
}
for (const entry of vciManager.jointNodes) {
    for (const joint of entry.joints) {
        // joint.nodeIndex is the connected body (-1 = world)
        const connected = vciManager.findTransformNode(joint.nodeIndex);
        console.log(entry.node.name, joint.type, connected?.name ?? 'world');
    }
}
```

## Performance

The per-frame update paths (LookAt / expressions / node constraints / spring bones / animation retargeting) are zero-allocation: all scratch vectors, quaternions and matrices are pooled at module scope, so driving an avatar does not create steady-state GC pressure. The constraint topological sort is cached after load, idle frames (no active expression) skip the morph/material update pass entirely, and MToon reuses a scratch vector for its per-bind `_Time` uniform.

Additional optimizations for heavy scenes:

```ts
import { optimizeMorphTargets, combineMorphs, removeUnnecessaryVertices } from 'babylon-vrm';

// Fix the morph target shader variant (Babylon.js 9 stores morph targets in a
// texture on WebGL2/WebGPU; setting a constant influencer count compiles a
// single vertex shader instead of recompiling whenever expressions activate
// or deactivate, e.g. blinking)
optimizeMorphTargets(vrm1Manager.rootMesh /* or vrmManager.rootMesh */);

// Reduce geometry and morph data memory (run once after load)
combineMorphs(vrm1Manager); // merge per-expression morph targets (VRM 1.0)
removeUnnecessaryVertices(vrm1Manager.rootMesh);
```

App-level tips that pair well with the loader:

- Set `mesh.isPickable = false` on avatar meshes (or `scene.skipPointerMovePicking = true`) unless you need pointer picking — VRM meshes are often high-poly.
- `engine.enableParallelShaderCompilation = true` keeps first-render shader compilation off the critical path.
- Dispose models with `manager.dispose()` followed by `deepDispose(rootMesh)` to release materials, textures and skeletons.

## Build

```s
# dist/ bundle (ESM JavaScript + .d.ts type declarations)
$ npm run build

# dist-tsc/ (tsc emit, standard-decorator JavaScript + raw shaders) for
# direct-browser debugging without a bundler
$ npm run build:tsc

Debugging with the sandbox

$ npm run dev

You can see the inspector on http://localhost:5173/test/index.html?inspector

Related Links

• BabylonJS/Babylon.js: Babylon.js: a complete JavaScript framework for building 3D games with HTML 5 and WebGL
• virtual-cast/babylon-vrm-loader: VRM loader for babylon.js (the original project this package is refactored from)
• vrm-c/UniVRM: Unity package that can import and export VRM format
• pixiv/three-vrm: Use VRM on Three.js (reference implementation of the VRM 1.0 features)
• virtual-cast/babylon-mtoon-material: Unity MToon Shader WebGL porting to babylon.js.

Licenses

see LICENSE.

This project is an AI-assisted refactor of virtual-cast/babylon-vrm-loader (MIT); its original copyright notice is preserved in the LICENSE.

This project uses babylon.js with Apache License, Version 2.0.

This project vendors babylon-mtoon-material with MIT License under src/mtoon-material/.