@npcify/web

Add a Npcify AI character to any browser application or Three.js 3D game. Includes VRM loading, humanoid skeleton and bone access, animation mixing, lip sync, facial expressions, Web Audio routing, live chat, baked conversations, and an optional managed canvas helper. No Vue or React dependency.

Install

npm install @npcify/web

Use an ESM-capable bundler, such as Vite. This package includes the required Npcify runtime code; consumers do not need private workspace packages.

Mode 1: Three.js Game Integration (loadCharacter)

Use loadCharacter (or new Character(...) / createCharacter(...)) when building a Three.js game or custom 3D scene and you want direct, real-time access to the character's scene, mesh, skinnedMeshes, materials, humanoid bones, audio element / Web Audio nodes, facial expressions, skeletal animation, and live speech state inside your own render loop.

import * as THREE from 'three'
import { loadCharacter } from '@npcify/web'

const scene = new THREE.Scene()
const camera = new THREE.PerspectiveCamera(35, window.innerWidth / window.innerHeight, 0.1, 100)
camera.position.set(0, 1.4, 2.5)

const renderer = new THREE.WebGLRenderer({ antialias: true })
renderer.setSize(window.innerWidth, window.innerHeight)
renderer.shadowMap.enabled = true
document.body.appendChild(renderer.domElement)

// Load the character and attach its VRM scene directly to your Three.js scene
const character = await loadCharacter({
  baseUrl: 'https://api.npcify.ai',
  characterId: 'YOUR_CHARACTER_ID',
  apiKey: 'YOUR_SCOPED_API_KEY',
  parent: scene,
  gazeTarget: camera,
  shadows: true,
  position: [0, 0, 0],
})

// Inspect meshes, materials, and humanoid bones in real time:
console.log(character.scene) // THREE.Group (vrm.scene)
console.log(character.mesh) // Primary THREE.SkinnedMesh | THREE.Mesh
console.log(character.skinnedMeshes) // readonly THREE.SkinnedMesh[]
console.log(character.materials) // readonly THREE.Material[]
const headBone = character.getBone('head') // THREE.Object3D | null

// Optional: route character voice into Three.js 3D spatial audio (PositionalAudio)
const listener = new THREE.AudioListener()
camera.add(listener)
const positionalAudio = new THREE.PositionalAudio(listener)
positionalAudio.setNodeSource(character.audio.createMediaElementSource(listener.context))
headBone?.add(positionalAudio)

// Start a conversation inside a user gesture handler:
startButton.onclick = async () => {
  await character.audio.unlock()
  await character.createConversation()
}

// Advance the character in your game loop and read real-time state:
const clock = new THREE.Clock()
function animate() {
  requestAnimationFrame(animate)
  const delta = Math.min(clock.getDelta(), 0.1)
  character.update(delta)

  // Real-time state on every frame:
  // character.speaking, character.revealedText, character.options
  // character.audio.playing, character.audio.currentTime, character.audio.volume
  // character.expressions.mouthOpenness, character.expressions.weights
  // character.animation.activeAnimationId, character.mixer

  renderer.render(scene, camera)
}
animate()

Character Real-Time Interface

• 3D Scene & Mesh (Character):
  • character.scene / character.object3d / character.root: Root THREE.Group (vrm.scene).
  • character.mesh: Primary THREE.SkinnedMesh | THREE.Mesh | null.
  • character.meshes: readonly THREE.Mesh[] across the character hierarchy.
  • character.skinnedMeshes: readonly THREE.SkinnedMesh[].
  • character.materials: Deduplicated readonly THREE.Material[].
  • character.skeletons: Deduplicated readonly THREE.Skeleton[].
  • character.humanoid: Three-VRM VRMHumanoid | null.
  • character.getBone(name) / character.getNormalizedBone(name): Look up a humanoid bone THREE.Object3D ('head', 'hips', 'rightHand', etc.).
  • character.attachTo(parent) / character.detach(): Attach or detach from any THREE.Object3D.
  • character.setShadows({ cast, receive }): Configure shadow flags across all character meshes.
  • character.setTransform({ position, rotation, scale }): Set root transform with Vector3/Euler or [x, y, z] tuples.
  • character.setGazeTarget(object3d): Make template-face eyes follow a THREE.Camera or THREE.Object3D with their gaze cone and idle eye moves. Other faces do not follow it. mountNpcify sets its camera.
  • character.lookAt(object3d | [x, y, z] | { x, y, z } | x, y, z): Look at an object or a world position. A template face keeps its gaze cone, idle eye moves and staging look cues; other faces turn their eyes with three-vrm vrm.lookAt.target. Coordinates that are not finite numbers throw a RangeError.
  • character.clearLookAt(): Remove the gaze target, so that the eyes look ahead.
• Audio & Web Audio (character.audio):
  • character.audio.element: Underlying HTMLAudioElement | null.
  • character.audio.playing, paused, ended, currentTime, duration, unlocked.
  • character.audio.volume (0..1), muted (boolean), playbackRate (number), crossOrigin.
  • character.audio.activeVisemes: Active lip-sync viseme timeline.
  • character.audio.createMediaElementSource(audioContext): Create or reuse a MediaElementAudioSourceNode for THREE.PositionalAudio, AnalyserNode, or custom Web Audio graphs.
  • character.audio.connectAudioNode(destination): Connect character voice directly to any Web Audio AudioNode.
  • character.audio.unlock() / character.audio.stop().
• Facial Expressions (character.expressions):
  • character.expressions.all, emotions, visemes: Available VRM expression channel names.
  • character.expressions.weights: Real-time map of applied channel weights (Record<string, number>).
  • character.expressions.mouthOpenness: Real-time mouth openness in [0, 1].
  • character.expressions.getWeight(name) / setWeight(name, weight).
  • character.expressions.play(name, { weight, durationMs }).
  • character.expressions.setEmotion(name, weight) / clearEmotion() / reset().
• Skeletal Animation (character.animation):
  • character.animation.availableIds, idleAnimationId, activeAnimationId.
  • character.animation.mixer: Underlying THREE.AnimationMixer | null.
  • character.animation.activeAction: Active THREE.AnimationAction | null.
  • await character.animation.play(animationId, { fadeDuration, loop }): Automatically fetches lazy animations if needed and cross-fades.
  • character.animation.playIdle(fadeDuration) / character.animation.stop(fadeDuration).
• Dialogue & Speech State:
  • character.speaking, character.thinking, character.currentSpeech, character.currentText, character.revealedText, character.options, character.cursor, character.speech.
  • await character.createConversation(), await character.sendMessage(text), await character.endConversation().
  • await character.dispose(): Ends conversation, detaches from the scene, removes the audio element, and disposes VRM GPU resources.

Mode 2: Managed Canvas (mountNpcify)

Create a key in your character's Export → API keys tab. Choose the characters that integration may access. Copy the key when created; it is shown only once.

<canvas id="npc" style="width: 100%; height: 480px; touch-action: none"></canvas>
<button id="start" disabled>Start conversation</button>
<div id="replies"></div>
<p id="status" role="status"></p>
<script type="module" src="/main.js"></script>

// main.js
import { mountNpcify } from '@npcify/web'

const start = document.querySelector('#start')
const replies = document.querySelector('#replies')
const status = document.querySelector('#status')
const view = await mountNpcify({
  canvas: document.querySelector('#npc'),
  baseUrl: 'https://api.npcify.ai',
  characterId: 'YOUR_CHARACTER_ID',
  apiKey: 'YOUR_SCOPED_API_KEY',
  serverRendering: false, // Use the setting from Export → Web SDK.
  onError: (error) => { status.textContent = error.message },
})
const { character } = view
character.onOptionsReady(({ options }) => {
  replies.replaceChildren()
  for (const option of options) {
    const button = document.createElement('button')
    button.textContent = option.label
    button.onclick = async () => {
      replies.replaceChildren()
      try { await character.sendMessage(option.label) }
      catch (error) { status.textContent = error.message }
    }
    replies.append(button)
  }
})
start.disabled = false
start.onclick = async () => {
  start.disabled = true
  try {
    await character.unlockAudio()
    await character.createConversation()
  } catch (error) {
    status.textContent = error.message
    start.disabled = false
  }
}
window.addEventListener('pagehide', () => { void view.dispose() }, { once: true })

Run the included examples/index.html, examples/main.js, or examples/three-game.js with any ESM bundler after replacing their configuration placeholders. Keys in browser code are visible to visitors. Use narrow scopes; keep keys in your backend when they must remain private. Calls made with a key are billed to its owner.

Runtime and lifecycle

• mountNpcify({ canvas, baseUrl, characterId, apiKey, serverRendering }) returns { character, npcify, scene, camera, renderer, controls, dispose } after loading. Give the canvas a nonzero CSS size.
• loadCharacter({ baseUrl, characterId, apiKey, parent, gazeTarget, shadows }) loads and returns a Character instance for custom Three.js scenes and render loops.
• Call unlockAudio() (or character.audio.unlock()) during a user gesture before starting playback.
• onOptionsReady delivers choices after playback finishes. sendMessage(label) selects a choice. Custom replies depend on the character configuration.
• onStagingResult delivers full results. onCharacterReveal supports subtitles. Every on* registration method returns an () => void unsubscribe function.
• dispose() stops rendering, releases GPU resources, and disconnects chat. Call it when navigating away or removing the character.
• serverRendering: true uses acknowledged WebP frames through an authenticated render session. Protected VRM/VRMA files stay on the server. Remount after a render connection error. This release does not use the hosted site's WebRTC video transport.
• Baked characters load their exported graph and run without runtime generation; baked playback requires browser rendering.
• Keys may cover multiple characters. Editing scopes, rotating, or deleting keys affects subsequent requests and disconnects associated conversation sockets.
• ESM only; require() is not a supported package entry point. Browser APIs are used when creating a view; do not call mountNpcify during server rendering.

Maintainer release

Bump the package version, merge the validated change into develop, then run Publish npm SDKs in GitHub Actions. Homelab builds, tests and packs the SDKs; a single minimal GitHub-hosted job publishes the prepared archives through npm Trusted Publishers. Homelab verifies the registry checksums afterward. See the release guide for the workflow and trust configuration.

Private Npcify dependencies are bundled into the ESM and declaration entry; Third-party renderer and transport dependencies remain normal npm dependencies.