TJBot Library (Node.js)

What is TJBot?

TJBot is an open-source robot created by IBM for learning how to program artificial intelligence applications. This library provides a simple, high-level interface to control TJBot running on a Raspberry Pi.

What Can TJBot Do?

TJBot's core capabilities are:

• Listen – Capture and transcribe speech with Speech-to-Text
• See – Recognize objects, faces, and image classes (and describe images with Azure Vision)
• Shine – Control an RGB LED in various colors and effects
• Speak – Play audio and synthesize speech with Text-to-Speech
• Wave – Move its arm using a servo motor

This library supports local AI backends (sherpa-onnx for speech, ONNX runtime for vision) and cloud services for speech and vision, including IBM Watson (speech), Google Cloud (speech + vision), and Microsoft Azure (speech + vision).

System Dependencies

Install additional system packages:

sudo apt-get install libgpiod-dev liblgpiod-dev rpicam-apps-lite

[!TIP] These packages are installed as part of TJBot's setup script.

Installation

Install the library from npm:

npm install --save tjbot-ce

[!TIP] The easiest way to create a new Node.js-based TJBot recipe is using the tjbot command: tjbot create [recipe-name], which will automatically set up node-tjbotlib as a dependency.

Quick Start

Importing & Instantiating TJBot

Use ES6 module syntax to import the TJBot class in your recipe. This example initializes TJBot's LED and servo motor.

import TJBot from 'tjbot-ce';

tj = TJBot.getInstance().initialize({
   hardware: {
      led: true,
      servo: true
   }
});

[!NOTE] TJBot uses a singleton pattern since there is only a single TJBot.

Example 1: Control an LED

This example initializes a NeoPixel LED on the GPIO 18 pin and sets it to various colors:

import TJBot from 'tjbot-ce';

const tj = TJBot.getInstance().initialize({
   hardware: {
      led: true
   },
   shine: {
      hasNeopixelLED: true,
      neopixel: {
         gpioPin: 18
      }
   }
});

// Set LED to red
await tj.shine('red');

// Set LED to green (using hexadecimal)
await tj.shine('#00FF00');

// Pulse the LED blue
await tj.pulse('blue');

Example 2: Speak Text using Text-to-Speech (TTS)

This example demonstrates how to make TJBot speak using on-device Text-to-Speech.

[!NOTE] The text-to-speech backend used by TJBot is set in TJBot's configuration file, located at ~/.tjbot/tjbot.toml. By default, TJBot uses the sherpa-onnx text-to-speech backend.

import TJBot from 'tjbot-ce';

const tj = TJBot.getInstance().initialize({
   hardware: {
      speaker: true
   },
   speak: {
      backend: {
         type: "local"
      }
   }
});

await tj.speak('Hello, I am T J Bot!');

[!IMPORTANT] Many Text-to-Speech models are not trained to pronounce "TJBot" the way it is written. Writing it out as "T J Bot" makes it easier for these models to pronounce it correctly.

Example 3: Change TJBot's Configuration

TJBot uses a cascading configuration system that loads settings from multiple sources in order of priority. First, default configuration settings are loaded from the tjbot.default.toml file that is bundled within node-tjbotlib. Next, user-specific configuration is loaded from your ~/.tjbot/tjbot.toml configuration file. Finally, recipe-specific configuration is loaded from the recipe.toml file in your recipe's directory (if present).

User configuration (~/.tjbot/tjbot.toml):

This file contains configuration settings for the hardware components of your TJBot, such as which pins the LED and servo are connected to, which audio devices to use for recording & playback, and which STT/TTS/Vision backends to use. Example:

[log]
level = 'debug' # TJBot will print a lot of detail about its operations to the console

[shine.neopixel]
gpioPin = 18 # GPIO 18 / Physical Pin 12

...

[!TIP] You can either use the tjbot config command to edit TJBot's configuration or you can edit the ~/.tjbot/tjbot.toml file directly.

Recipe-specific configuration (recipe.toml):

This file contains configuration settings for your recipe. It is placed in your project directory.

tjbot_name = "tinker"
favorite_color = "blue"
cloud_api_key = "xyzabc"

Recipe-specific settings are loaded using the TJBot.getRecipeConfig() class method.

import TJBot from 'tjbot-ce';

// read recipe-specific config
const config = TJBot.getRecipeConfig();

const tj = TJBot.getInstance().initialize({
   hardware: {
      led: true
   }
});

const favorite_color = config.favorite_color as string;
tj.shine(favorite_color);

Using override_config for Runtime Configuration

You can also pass specific configuration requirements directly to the TJBot.initialize() method using the overrideConfig parameter. This configuration merges with the cascaded defaults (not replaces them):

import TJBot from 'tjbot-ce';

const config = {
    shine: {
        hasNeopixelLED: true,
        neopixel: {gpioPin: 18}
    }
};

tj = TJBot.getInstance().initialize(config);

// The final config is: defaults + ~/.tjbot/tjbot.toml + override_config

[!TIP] You can use overrideConfig to explicitly enable any specific hardware required for your recipe.

Configuration Reference

TJBot uses TOML for its configuration. The canonical default configuration lives in vendor/tjbot-config/tjbot.default.toml and follows the schema specified in vendor/tjbot-config/tjbot-config.schema.yaml.

These files are synced into src/config/ during builds. They can be synced manually by running this command:

npm run sync:config-schema

Custom Models & Model Registry

TJBot ships with a built-in model registry in vendor/tjbot-config/model-registry.yaml. The registry is synced into src/config/model-registry.yaml during builds for local development and packaging. You can register additional ML models in your ~/.tjbot/tjbot.toml file. Search for the section titled "On-Device ML Models".

Example: register a custom vision classification model and use it locally:

[[models]]
type = 'vision.classification'
key = 'my-classifier'
label = 'My Classifier'
url = 'file:///home/pi/models/my-classifier.zip'
folder = 'my-classifier'
kind = 'classification'
required = ['model.onnx', 'labels.txt']
labelUrl = 'file:///home/pi/models/labels.txt'
inputShape = [1, 3, 224, 224]

[see.backend]
type = 'local'

[see.backend.local]
imageClassificationModel = 'my-classifier'

You can register custom speech models in the same way.

API Documentation

For detailed API documentation, method signatures, and advanced usage, visit the TJBot Node.js SDK Reference.

Testing

The library uses Vitest for testing with two tiers of tests:

Automated Tests (Core Tests)

TJBot ships with a number of unit tests that verify the library's core functionality.

# Run all automated tests
npm run test

# Run tests with coverage report
npm run test:coverage

These tests run on a Raspberry Pi but do not require any specific TJBot hardware.

[!WARNING] TJBot's software has not been tested on operating systems or hardware other than Raspian OS on Raspberry Pi.

Interactive Hardware Tests (Live Tests)

TJBot also ships with a number of interactive tests meant to test (and debug) your Raspberry Pi hardware setup. These tests validate each of these components:

# Test the camera
npm run test-camera

# Test the LED
npm run test-led

# Test the microphone
npm run test-microphone

# Test the servo
npm run test-servo

# Test the speaker
npm run test-speaker

# Test the STT service (allows you to choose which backend to use)
npm run test-stt

# Test the TTS service (allows you to choose which backend to use)
npm run test-tts

# Test the Vision service (allows you to choose which backend to use and which vision task to perform)
npm run test-vision

[!WARNING] These tests must be run on a Raspberry Pi with properly connected hardware components.

Development

To set up a local development environment, you will first need to check out node-tjbotlib from GitHub. Then you will create a new recipe and point it to your locally-checked out copy of node-tjbotlib.

Clone node-tjbotlib

1. Clone the repository

   git clone --recurse-submodules https://github.com/tjbot-ce/node-tjbotlib.git
   cd node-tjbotlib

   If you already cloned the repo without submodules, run:

   git submodule update --init --recursive

   This initializes the shared TJBot configuration assets submodule at vendor/tjbot-config.

2. Install dependencies

   npm install

3. Build the TypeScript code

   npm run build

4. Link the repository for npm

   npm link

5. Run tests

   npm run test

6. Lint and format code

   npm run lint
   npm run format

Create a Recipe

Create a new recipe using tjbot create <recipe>, then link it to the local version of node-tjbotlib.

1. Create a new recipe

   tjbot create my_recipe

2. Link the recipe to the local node-tjbotlib

   cd my_recipe
   npm link tjbot

3. Install dependencies

   npm install

Troubleshoot

If you are having difficulties in making your TJBot work, please see the troubleshooting guide.

Contribute

If you would like to contribute to TJBot, please see the contributor's guide.

License

This project is licensed under Apache 2.0. Full license text is available in LICENSE.