Cboard AI Engine

This engine powers the Cboard AI builder, designed to generate content suggestions for communication boards and create new pictograms as necessary.

With a simple prompt, it will generate a list of suggestions that can be used to create an AAC board. Each suggestion will be associated with a text description and a list of images.

The images are retrieved from the Global Symbols website, and the text descriptions are generated using the OpenAI Node API Library. If the image is not found, the engine will create a new pictogram using the Cboard Pictonizer.

Table of Contents

• Installation
• Usage
  • Initialization
• Types
• Methods
  • getSuggestionsAndProcessPictograms
  • getSuggestions
  • pictonizer
• Error Handling
• License

Installation

npm install cboard-ai-engine

or

yarn add cboard-ai-engine

Usage

The code below shows how to get started using the Cboard AI Engine.

import { initEngine } from "cboard-ai-engine";

const engineInstance = initEngine({
  openAIConfiguration,
  globalSymbolsApiURL,
  pictonizerConfiguration,
});

const suggestions = await engineInstance.getSuggestionsAndProcessPictograms({
  prompt: "Brazilian food",
  maxSuggestions: 5,
  symbolSet: "arasaac",
  language: "eng",
});

Initialization

const engineInstance = initEngine({
  openAIConfiguration,
  globalSymbolsApiURL,
  pictonizerConfiguration,
});

The initEngine function is used to initialize the engine. Takes an object with the following properties as its only argument:

• openAIConfiguration: Object with the OpenAI configuration. Required.

const openAIConfiguration = {
  apiKey: "your openai api key",
  basePath: "https://your-openai-base-path.com",
  baseOptions: {
    headers: { "api-key": "your openai api key" },
    params: { "api-version": "2022-12-01" },
  },
};

• globalSymbolsApiURL: The Global Symbols API URL. Default is https://www.globalsymbols.com/api/v1/labels/search/. Optional.

• pictonizerConfiguration: Object with the Pictonizer configuration. Optional. If not passed, the pictonizer() method will return a "empty" image. For more info see pictonizer.

import { type PictonizerConfiguration } from "cboard-ai-engine";

const pictonizerConfiguration = {
    URL: process.env.PICTONIZER_URL, 
    token: process.env.PICTONIZER_AUTH_TOKEN,
    keyWords: 'arasaac pictograms',
} as PictonizerConfiguration;

Return:

It returns an instance of the engine with the following methods:

• getSuggestionsAndProcessPictograms: This method is used to get the suggestions and process the pictograms. It will first search for the images in Global Symbols, and if no image is found, it will generate a new one with the Cboard Pictonizer.

  It returns a list of suggestions that can be used to create an AAC board. Each one is associated with a text description and a pictogram.

• getSuggestions: This method is used to get suggestions with images solely from Global Symbols.

• pictonizer: This method is used to generate a new pictogram using the Cboard Pictonizer.

Types

The engine uses the following types:

export type Suggestion = {
  id: string; // Unique identifier for the suggestion
  label: string; // The text description of the suggestion
  locale: string; // The language of the suggestion
  pictogram: { 
    isAIGenerated: boolean; // If the pictogram was generated by the AI
    images:
      | {
          id: string; // Indentifier for the image from Global Symbols
          symbolSet: string; // The symbol set of the image
          url: string; // The URL of the image
        }[]
      | AIImage[]; 
  };
};

export type AIImage = {
  blob: Blob | null; // The image blob
  ok: boolean; // If the image was generated successfully 
  error?: string; // The error message if the image was not generated
  prompt: string; // The prompt used to generate the image
};

Methods

getSuggestionsAndProcessPictograms

async function getSuggestionsAndProcessPictograms({
  prompt,
  maxSuggestions,
  symbolSet,
  language = DEFAULT_LANGUAGE,
}: {
  prompt: string;
  maxSuggestions: number;
  symbolSet?: string;
  language: string;
}): Promise<Suggestion[]>;

This method is used to get the suggestions and process the pictograms. It will first search for the images in Global Symbols, and if no image is found, it will generate a new one with the Cboard Pictonizer.

It returns a list of suggestions that can be used to create an AAC board. Each one is associated with a text description and a pictogram.

Parameters:

• prompt : The prompt to be used to generate the suggestions. Required. Type: string.

• maxSuggestions: The maximum number of suggestions to be returned. Default is 10. Optional. Type: number.

• symbolSet: The symbol set to be used. If undefined, images will be searched across all Global Symbol image banks. Optional. Type: string.

• language: The language to be used. Default is eng. Optional. Type: string.

Return:

It returns an array of Suggestion.

getSuggestions

async function getSuggestions({
  prompt,
  maxSuggestions,
  symbolSet,
  language = DEFAULT_LANGUAGE,
}: {
  prompt: string;
  maxSuggestions: number;
  symbolSet?: string;
  language: string;
}): Promise<Suggestion[]>;

This method is used to get the suggestions with images solely from Global Symbols. It will not generate new pictograms with the Cboard Pictonizer.

Parameters:

• prompt : The prompt to be used to generate the suggestions. Required. Type: string.

• maxSuggestions: The maximum number of suggestions to be returned. Default is 10. Optional. Type: number.

• symbolSet: The symbol set to be used. If undefined, images will be searched across all Global Symbol image banks. Optional. Type: string.

• language: The language to be used. Default is eng. Optional. Type: string.

Return:

It returns an array of Suggestion.

pictonizer

const pictogram = await engineInstance.pictonizer(imagePrompt: string) => Promise<AIImage>;

This method is used to generate a new pictogram using the Cboard Pictonizer.

Parameters:

• imagePrompt : The prompt to be used to generate the pictogram. Required. Type: string.

Return:

It returns an AIImage object.

const AIImage = {
    blob: Blob;
    ok: true;
    error: undefined;
    prompt: "Brazilian food arasaac pictograms"; 
}

// NOTE: The prompt is a concatenation of the prompt
// and the keyWords from the PictonizerConfiguration.

If no URL or Token is passed on the PictonizerConfiguration parameter, or occurs an error while generating. It will return:

const AIImage = {
    blob: null;
    ok: false;
    error: "ERROR: Can't generate image";
    prompt: "Brazilian food arasaac pictograms"; 
}

// NOTE: The prompt is a concatenation of the prompt
// and the keyWords from the PictonizerConfiguration.

And no error will be thrown.

Error Handling

When an error occurs, an error will be thrown. It is recommended to use a try/catch block to handle it.

try {
  const suggestions = await engineInstance.getSuggestionsAndProcessPictograms({
    prompt,
    maxSuggestions,
    symbolSet,
    language
  });
} catch (error) {
  console.error(error);
}

NOTE: Is not needed on the initialization method.

License

Copyright © 2024 Cboard

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License version 3 as published by the Free Software Foundation.

• Code - GPLv3
• ARASAAC Symbols - CC BY-NC-SA 4.0