Animated 3D Avatar for JavaScript Applications

This is a npm module to install in your web environment and then use in JavaScript. With this module you can use the

Set up npm

Install npm and node if not already available on your system. You can check this with

$ npm --version
$ node --version

If not already installed, install with

$ sudo apt install nodejs
$ sudo apt-install npm

Make sure your node version is >= 14. To update node.js on Ubuntu: (for newest version check here: https://nodejs.org/en/download/package-manager)

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
$ source ~/.nvm/nvm.sh
$ nvm install 20

Install required packages for npm with this command run in the project/root directory:

$ npm install

Starting for development

Initial Setup

In the project/root directory run this command to build

$  npm run build

in /test/ are the project files to host a demo. Initial setup:

$ cd test
$ npm i

Deployment

It's best to open a new terminal in /test/ and then host the demo using this command.

$ npx vite --port 8080

Changes to the demo are updates automatically. To apply module changes run

$  npm run build

Publishing new version of the package

Make sure you have set our company registry in the npm config. Replace with your token.

$ npm config set -- //gitlab.lingucity.de/:_authToken=<token>

Update the version of the package in package.json and package-lock.json. Only the root directory ones! Do not do this in /tests/.

Build changes

$ npm run build

Publish new package. Must change version in project.json first!

$ npm publish

Usage

Simple Setup

If you are importing this module to use in your project, this is how you use it:

1. Configure npm

Make sure you have set our company registry in the npm config. Replace with your token.

$ npm config set -- //gitlab.lingucity.de/:_authToken=<token>

2. Install package(s)

Install the package with npm and make sure all dependencies are installed.

$ npm install @linguwerk/3d_web_avatar

If there's any strange behavior, you might need to check your three.js and tween.js versions. These are the dependencies in package.json we use for development:

"dependencies": {
  "@tweenjs/tween.js": "^17.3.5",
  "three": "^0.162.0"
}

If you don't have them installed, or have the wrong version installed, use the above dependencies or install using these commands:

$ npm install three@^0.162.0
$ npm install tween@^17.3.5

3. Import and use avatar

In javascript, import the robot and init your avatar:

import { Robot } from "@linguwerk/3d_web_avatar";

let myAvatar;
const avatarContainerId = "containerForAvatar";
const audioPlayerID = "audioPlayer";

function createAvatar() {
    myAvatar = new Robot(avatarContainerId, audioPlayerID);
    myAvatar.setBackgoundColor(null); // clear background
    myAvatar.camera.fov = 24; // to get avatar closer/further away
}

function animate(animName) {
    if (animName != null) {
        myAvatar.playAnimation(animName);
    }
}
function setSentiment(sentiment) {
    myAvatar.setSentiment(sentiment);
}

createAvatar();

If you have issues caused by the dependent packages not being found by this module, here is a workaround to copy-paste at the top of your JavaScript file:

import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
import TWEEN from '@tweenjs/tween.js';
import { DRACOLoader } from 'three/addons/loaders/DRACOLoader.js';
import Stats from 'stats.js'

window.THREE = THREE;
window.GLTFLoader = GLTFLoader;
window.OrbitControls = OrbitControls;
window.TWEEN = TWEEN;
window.DRACOLoader = DRACOLoader;
window.Stats = Stats;

If you have issues using the avatar in other JavaScript files outside you main, consider copy-pasting this workaround at the bottom of your main JavaScript file:

window.myAvatar = myAvatar;    
window.animate = animate;
window.setSentiment = setSentiment;

Dokumentation

TODO: Add all functionality available to the user.

Avatar
    .getAllAnimations()
    .playAnimation(animName)
    .playAction(actionName)
    .setSentiment(sentiName)
    .setBackgoundColor(color)
    .setZoomFactor(number)

Advanced Usage

There are many modifications you can make beyond the specified functions if you are willing to get into three.js. With the Avatar you can access the following to make three.js modifications:

// Set things like background color
myAvatar.scene

// Set things like fov
myAvatar.camera

// 3d model loaded in with three.js
myAvatar.gltf

// The renderer
myAvatar.renderer

// The materials configured in config
myAvatar.materials[<materialName>]

// Set things like rotation, position, material, ...
myAvatar.sceneItems[<seneItemName>].sceneItem

// action clipped to the animationMixer from glts file + Code actions
myAvatar.animator.animations[<animName>].actions[<idx>].action
myAvatar.animator.actions[<actionName>].action

Using your own 3D Model

To use your own 3D Model, you must specify a directory in which all the required files are located. Your 3D Model will need:

• A gltf file containing
  • the 3D model
  • it's rig
  • keyframe animations
  • materials
  • ...
• A config file in which you specify
  • the 3D objects your model is made of (sceneitems)
  • Your keyframe actions in the gltf file
  • Animations you want to have available (add your actions to define what should be executed)
  • Your materials if you want to alter them after
• If you want to have code actions and apply sentiments, you must write a class for your 3D model that extends Avatar and implement it analogous to Robot in robot.js

Then you can just specify the directory and name of your model when creating the avatar:

let config = {// your loaded config as JSON};
let modelDirectory = /path/to/your/modelDirectory/;
let modelName = "name-of-your-model-file";
let myAvatar = new Avatar(avatarContainerId, audioPlayerID, config, modelDirectory, modelName);