Sprucebot LLM

A TypeScript library for leveraging large language models to do... anything!

• Has memory
  • Remembers past messages to build context
  • Configure how much of the conversation your bot should remember
• Manages state
  • The state builds as the conversation continues
  • Invoke callbacks whenever state changes
• Connect to 3rd party APIs
  • Pull in data in real time
  • Have your bot respond generated responses
• Unlimited use cases
  • Skill architecture for extensibility
  • Leverage Skills to get your bot to complete any task!
  • Swap skills mid-conversation to chain multi-step flows
• Multiple adapter support
  • OpenAI - GPT-4o, o1, and other OpenAI models
  • Anthropic - Claude models with prompt caching support
  • Ollama - Run local models like Llama, Mistral, etc.
  • Custom adapters - Implement your own
• Fully typed
  • Built in modern TypeScript
  • Fully typed schema-based state management (powered by @sprucelabs/schema)

Lexicon

• Bot: This is the abstraction that holds the agent's personality.
• Skill: Describes what the Bot is responsible for doing and how to do it (including tool integration).

Getting started

Install the library as a dependency

yarn add @sprucelabs/sprucebot-llm

npm install @sprucelabs/sprucebot-llm

Cloning and testing directly

To clone the repository and prepare for development, do the following:

git clone https://github.com/sprucelabsai/sprucebot-llm.git
cd sprucebot-llm
yarn rebuild
code .

Testing it out for yourself

You can use sprucebot-llm inside any JavaScript runtime (Node.js, Bun, browser).

If you want to try this locally, you can checkout chat.ts. Create a .env file with your OpenAI API key first:

OPEN_AI_API_KEY=your_api_key_here

Here are the contents of that file for you to review now, rather than needing to explore the codebase.

import { stdin as input, stdout as output } from 'node:process'
import * as readline from 'node:readline/promises'
import dotenv from 'dotenv'
import OpenAiAdapter from './bots/adapters/OpenAi'
import SprucebotLlmFactory from './bots/SprucebotLlmFactory'
import buildCallbackSkill from './examples/buildCallbackSkill'
import buildFileTransformerSkill from './examples/buildFileTransformerSkill'
import buildJokeSkill from './examples/buildJokeSkill'
import buildProfileSkill from './examples/buildProfileSkill'
import buildReceptionistSkill from './examples/buildReceptionistSkill'
dotenv.config({ quiet: true })

const rl = readline.createInterface({ input, output })

void (async () => {
    console.clear()

    // Create the adapter that handles actually sending the prompt to an LLM
    const adapter = OpenAiAdapter.Adapter(process.env.OPEN_AI_API_KEY!)

    // The LlmFactory is a layer of abstraction that simplifies bot creation
    // and enables test doubling (mocks, spies, etc)
    const bots = SprucebotLlmFactory.Factory(adapter)

    // Different examples of things you may want to play with (see line 34)
    const skills = {
        jokes: buildJokeSkill(bots),
        profile: buildProfileSkill(bots),
        callbacks: buildCallbackSkill(bots),
        fileTransformer: buildFileTransformerSkill(bots),
        receptionist: buildReceptionistSkill(bots),
    }

    // Construct a Bot and pass the skill of your choice
    const bot = bots.Bot({
        skill: skills.callbacks, //<-- try jokes, profile, etc.
        youAre: "a bot named Sprucebot that is in test mode. At the start of every conversation, you introduce yourself and announce that you are in test mode so I don't get confused! You are both hip and adorable. You say things like, 'Jeepers' and 'Golly' or even 'Jeezey peezy'!",
    })

    do {
        // Read from stdin
        const input = await rl.question('Message > ')

        // Send the message to the bot and log the response
        // We use a callback because one message may trigger a conversation
        // which will include many messages and bot.sendMessage(...) only
        // returns the last message send back from the LLM
        await bot.sendMessage(input, (message) => console.log('>', message))
    } while (!bot.getIsDone())

    console.log('Signing off...')
    rl.close()
})()

Message history and context limits

There are two different limits to be aware of:

• SprucebotLlmBotImpl.messageMemoryLimit (default: 10) controls how many messages are kept in the in-memory history on the Bot. Once the limit is hit, old messages are dropped and can no longer be sent to any adapter.
• OPENAI_MESSAGE_MEMORY_LIMIT or OpenAiAdapter.setMessageMemoryLimit(limit) controls how many of those tracked messages are included when sending a request to OpenAI. 0 (the default) means "no additional limit" beyond the Bot history.

To change the Bot history limit:

import { SprucebotLlmBotImpl } from '@sprucelabs/sprucebot-llm'

SprucebotLlmBotImpl.messageMemoryLimit = 20

Additional OpenAI context controls:

• OPENAI_PAST_MESSAGE_MAX_CHARS omits past (non-latest) messages longer than the limit, replacing them with [omitted due to length].
• OPENAI_SHOULD_REMEMBER_IMAGES=false omits images from older messages to save context, keeping only the most recent image and replacing older ones with [Image omitted to save context].

Adapters

OpenAI adapter configuration

Required environment variable:

OPEN_AI_API_KEY=your_api_key_here

Optional environment variables:

OPENAI_MESSAGE_MEMORY_LIMIT=0
OPENAI_PAST_MESSAGE_MAX_CHARS=0
OPENAI_SHOULD_REMEMBER_IMAGES=true
OPENAI_REASONING_EFFORT=low

Runtime configuration options:

const adapter = OpenAiAdapter.Adapter(process.env.OPEN_AI_API_KEY!, {
	log: console,
	model: 'gpt-4o',
	memoryLimit: 10,
	reasoningEffort: 'low',
	baseUrl: 'https://custom-endpoint/v1', // Optional custom base URL
})

// Or set after creation
adapter.setModel('gpt-4o')
adapter.setMessageMemoryLimit(10)
adapter.setReasoningEffort('low')

OpenAI adapter API

OpenAiAdapter exposes the following API:

• OpenAiAdapter.Adapter(apiKey, options?): create an adapter instance. Options include:
  • log - any logger that supports .info(...)
  • model - default model (e.g., 'gpt-4o')
  • memoryLimit - message memory limit
  • reasoningEffort - for reasoning models ('low', 'medium', 'high')
  • baseUrl - custom API endpoint
• adapter.setModel(model): set a default model for all requests unless a Skill overrides it.
• adapter.setMessageMemoryLimit(limit): limit how many tracked messages are sent to OpenAI.
• adapter.setReasoningEffort(effort): set reasoning_effort for models that support it.
• OpenAiAdapter.OpenAI: assign a custom OpenAI client class (useful for tests).

Requests are sent via openai.chat.completions.create(...) with messages built by the adapter from the Bot state and history.

Anthropic adapter

Use Claude models from Anthropic. Requires @anthropic-ai/sdk and an Anthropic API key.

import { AnthropicAdapter, SprucebotLlmFactory } from '@sprucelabs/sprucebot-llm'

const adapter = AnthropicAdapter.Adapter(process.env.ANTHROPIC_API_KEY!, {
    maxTokens: 4096,           // required
    model: 'claude-sonnet-4-5', // default
    log: yourLogger,           // optional
    memoryLimit: 10,           // optional
    thinking: false,           // optional: enable extended thinking mode
})

const bots = SprucebotLlmFactory.Factory(adapter)

Anthropic adapter options:

| Option | Type | Required | Description | |--------|------|----------|-------------| | maxTokens | number | yes | Maximum tokens for the model response | | model | string | no | Model to use (default: 'claude-sonnet-4-5') | | log | Log | no | Optional logger instance | | memoryLimit | number | no | Limit how many tracked messages are sent | | thinking | boolean | no | Enable extended thinking (thinking: adaptive) |

Anthropic prompt caching

The Anthropic adapter automatically enables prompt caching by inserting an ephemeral cache breakpoint after the system prompt. This allows Anthropic to cache the static portion of the prompt (your youAre + skill instructions) and only re-process the changing chat history on each turn — reducing latency and cost on long conversations.

Token usage (including cache creation and cache read tokens) is logged at the info level on each request:

[TOKEN USAGE] input=1234 cache_create=800 cache_read=400 output=256

No configuration is required — caching is applied automatically.

Ollama adapter

Run local models using Ollama. No API key required - just have Ollama running locally.

import { OllamaAdapter, SprucebotLlmFactory } from '@sprucelabs/sprucebot-llm'

// Create adapter for local Ollama instance
const adapter = OllamaAdapter.Adapter({
    model: 'llama2',      // or 'mistral', 'codellama', etc.
    log: console,         // optional logger
})

const bots = SprucebotLlmFactory.Factory(adapter)

const bot = bots.Bot({
    youAre: 'a helpful assistant',
    skill: bots.Skill({
        yourJobIfYouChooseToAcceptItIs: 'to answer questions'
    })
})

await bot.sendMessage('Hello!')

Ollama adapter options:

| Option | Type | Default | Description | |--------|------|---------|-------------| | model | string | - | Which Ollama model to use | | log | Log | - | Optional logger instance |

The Ollama adapter connects to http://localhost:11434/v1 by default (Ollama's OpenAI-compatible endpoint).

Custom adapters

You can bring your own adapter by implementing the LlmAdapter interface and passing it to SprucebotLlmFactory.Factory(...):

import {
	LlmAdapter,
	SprucebotLlmBot,
	SprucebotLlmFactory,
} from '@sprucelabs/sprucebot-llm'

class MyAdapter implements LlmAdapter {
	async sendMessage(bot: SprucebotLlmBot) {
		// Build your prompt from the bot's serialized state or messages
		const { messages } = bot.serialize()
		// Send to your model and return the model response as a string
		return `echo: ${messages[messages.length - 1]?.message ?? ''}`
	}
}

const bots = SprucebotLlmFactory.Factory(new MyAdapter())

Skills & State

Adding state to your conversation

This library depends on @sprucelabs/schema to handle the structure and validation rules around your state.

const skill = bots.Skill({
	yourJobIfYouChooseToAcceptItIs:
		'to collect some information from me! You are a receptionist with 20 years experience and are very focused on getting answers needed to complete my profile',
	model: 'gpt-4o', // Optional: override adapter's default model
	stateSchema: buildSchema({
		id: 'profile',
		fields: {
			firstName: {
				type: 'text',
				label: 'First name',
			},
			lastName: {
				type: 'text',
				label: 'Last name',
			},
			favoriteColor: {
				type: 'select',
				options: {
					choices: [
						{ label: 'Red', value: 'red' },
						{ label: 'Blue', value: 'blue' },
						{ label: 'Green', value: 'green' },
					],
				},
			},
		},
	})

Listening for state changes

If you supply a stateSchema, then your bot will work with it based on the job you decide to give it. While the conversation is taking place, if the state changes:

• If the state is on the Bot (you passed stateSchema to bots.Bot(...)), the Bot emits did-update-state.
• If the state is on the Skill (you passed stateSchema to bots.Skill(...)), the Skill emits did-update-state.

await skill.on('did-update-state', () => {
	console.log('we are making progress!')
	console.log(JSON.stringify(this.skill.getState()))
})

Pulling from 3rd party APIs

The approach to integrating 3rd party APIs (as well as dropping in other dynamic data into responses) is straightforward.

In this contrived example, you can see where you'd implement the callbacks for availableTimes, favoriteColor, and book to actually call the APIs and return the results.

const skill = bots.Skill({
	yourJobIfYouChooseToAcceptItIs:
		"to be the best appointment taker on the planet. You have a many years of experience. You are going to ask me only 2 questions for this practice run. First, you'll ask me to pick an available time. Then, you'll ask me to pick my favorite color (make sure to call the api to see what times and colors i can choose from). After all is said and done, make sure to actually book the appointment!:",
	weAreDoneWhen: 'the appointment is booked!',
	pleaseKeepInMindThat: [
		'people don\'t always know what they want, so be patient and guide them through the process.',
		'We have cancelled our coloring services, so if you\'re asked about them, tell the user we\'ve discontinued them.'
	],
	callbacks: {
		availableTimes: {
			cb: async () => {
				return [
					'9am',
					'10am',
					'11am',
					'1pm',
					'4pm',
					'5pm',
					'12am.',
				].join('\n')
			},
			useThisWhenever: 'your are showing what times i can pick from.',
		},
		favoriteColor: {
			cb: async () => {
				return ['red', 'blue', 'green', 'purple'].join('\n')
			},
			useThisWhenever:
				'your are showing what colors i can pick from.',
		},
		book: {
			cb: async (options) => {
				console.log('BOOKING OPTIONS', options)
				return 'Appointment booked!'
			},
			useThisWhenever: 'You are ready to book an appointment!',
			parameters: [
				{
					name: 'time',
					isRequired: true,
					type: 'string',
				},
				{
					name: 'color',
					isRequired: true,
					type: 'string',
				},
			],
		},
	},
})

Callback invocation format

The model is instructed to invoke callbacks using the following syntax (V2 format):

@callback { "name": "callbackName", "options": {} }

Multiple callbacks can be included in a single response, one per line. JSON must be on a single line — do not use multi-line or formatted JSON.

As a shorthand, you can also invoke a named callback directly:

@myCallback { "param": "value" }

Callbacks can return either a string or an image message shaped like { imageBase64, imageDescription } (see the "Sending images" section below).

Callback parameters can include basic types (e.g. text, number, boolean, dateMs, dateTimeMs) and select fields with choices from @sprucelabs/schema.

Note: This is not MCP (Model Context Protocol). MCP is focused on making APIs available to LLMs. sprucebot-llm comes at this from the opposite direction. It does not require you to do anything server-side, so you can connect to all your existing endpoints/tools/systems without needing to change them.

Sending images

You can send images to the bot by passing a base64-encoded image and a short description:

await bot.sendMessage({
	imageBase64: base64Png,
	imageDescription: 'A photo of a sunset over the mountains.',
})

There is a working example at src/chatWithImages.ts, and you can run it after building with:

yarn chat.images

Choosing a model

When you configure a Skill for a bot, you can specify the model that the skill will use. In other words, you can have different skills use different models depending on their requirements. The OpenAI adapter defaults to gpt-4o, and a Skill model (if set) overrides the adapter default.

const bookingSkill = bots.Skill({
	model: 'gpt-4o',
	yourJobIfYouChooseToAcceptItIs: 'to tell knock knock jokes!',
	pleaseKeepInMindThat: [
		'our audience is younger, so keep it PG!',
		'you should never laugh when someone does not get the joke.',
		"after each joke, you should tell me how many jokes you have left to tell before we're done.",
		'you should acknowledge if someone laughs at your joke by saying "Thanks!" or "Glad you thought that was funny"!',
	],
	weAreDoneWhen: 'you have told 3 jokes!',
})

const bookingBot = bots.Bot({
	skill: bookingSkill,
	youAre: "a bot named Sprucebot that is in test mode. At the start of every conversation, you introduce yourself and announce that you are in test mode so I don't get confused! You are both hip and adorable. You say things like, 'Jeepers' and 'Golly' or even 'Jeezey peezy'!",
})

If you are using reasoning models that accept reasoning_effort, you can set it via OPENAI_REASONING_EFFORT or adapter.setReasoningEffort(...).

Swapping Skills

You can replace the active skill on a bot at any time — even mid-conversation — by calling bot.setSkill(newSkill). This is useful for multi-step flows where each phase of the conversation has a different job, state schema, or set of callbacks.

const greetingSkill = bots.Skill({
    yourJobIfYouChooseToAcceptItIs: 'to greet the user and ask for their name',
    weAreDoneWhen: 'you have collected the user\'s name',
    stateSchema: buildSchema({
        id: 'greeting',
        fields: {
            firstName: { type: 'text', label: 'First name' },
        },
    }),
})

const bookingSkill = bots.Skill({
    yourJobIfYouChooseToAcceptItIs: 'to book an appointment for the user',
    weAreDoneWhen: 'the appointment is booked',
    callbacks: {
        book: {
            cb: async (options) => {
                await bookAppointment(options)
                return 'Appointment booked!'
            },
            useThisWhenever: 'you are ready to book the appointment',
        },
    },
})

const bot = bots.Bot({
    youAre: 'a helpful scheduling assistant',
    skill: greetingSkill,
})

// Listen for the greeting phase to complete, then swap to booking
await greetingSkill.on('did-update-state', async () => {
    if (greetingSkill.getState()?.firstName) {
        bot.setSkill(bookingSkill)
    }
})

Pre-populating state before swapping

Skills manage their own state independently of the bot. You can create a skill, set its initial state, and then hand it to the bot — useful when you already know some context going into the next phase:

const bookingSkill = bots.Skill({
    yourJobIfYouChooseToAcceptItIs: 'to book an appointment',
    weAreDoneWhen: 'the appointment is booked',
    stateSchema: buildSchema({
        id: 'booking',
        fields: {
            guestName: { type: 'text', label: 'Guest name' },
            time: { type: 'text', label: 'Appointment time' },
        },
    }),
})

// Pre-load state from the previous phase before the bot ever uses this skill
await bookingSkill.updateState({
    guestName: greetingSkill.getState()?.firstName,
})

// Now swap — the LLM will already know guestName when it starts
bot.setSkill(bookingSkill)

You can also read and update a skill's state at any point, regardless of whether it's active:

// Check what the skill has collected so far
console.log(bookingSkill.getState())
// { guestName: 'Taylor', time: undefined }

// Inject external data directly (e.g. from your own API)
await bookingSkill.updateState({ time: '10am' })

updateState merges — it only overwrites the fields you pass, leaving others intact. And since skills are plain objects, you can maintain references to them alongside the bot and keep them in sync independently.

Key behaviors when swapping skills

getIsDone() resets to false — swapping in a new skill always marks the bot as not done, so the conversation continues even if the previous skill had completed:

bot.markAsDone()
console.log(bot.getIsDone()) // true

bot.setSkill(bookingSkill)
console.log(bot.getIsDone()) // false — ready for the next phase

Message history is preserved — the full conversation history carries over. Only the active skill (its instructions, state schema, callbacks, and model) changes:

// Before swap: greeting skill active
await bot.sendMessage('Hi there!')

// Swap to a new skill mid-conversation
bot.setSkill(bookingSkill)

// After swap: booking skill active, but previous messages still in history
await bot.sendMessage('I\'d like to book for Tuesday')

The new skill's state replaces the old one — after a swap, bot.serialize().skill reflects the new skill's full configuration:

const skill2 = bots.Skill({
    stateSchema: carSchema,
    yourJobIfYouChooseToAcceptItIs: 'to help the user pick a car',
})

bot.setSkill(skill2)

// bot.serialize().skill now reflects skill2's schema and state
console.log(bot.serialize().skill)

Serialization and Persistence

You can snapshot a bot's full state — including message history, skill configuration, and any accumulated state — and later restore it. This is useful for persisting conversations across process restarts, saving/loading sessions, or transferring bot state.

// Save bot state (e.g. to a database or file)
const snapshot = bot.serialize()
// snapshot: { youAre, stateSchema, state, messages, skill }

// Later, recreate the bot and restore state
const bot2 = bots.Bot({
    skill: mySkill,
    youAre: 'a helpful assistant',
})
bot2.unserialize(snapshot)
// bot2 now has the same message history and state as bot had when serialized

The skill's state is also preserved through serialization:

const skillSnapshot = skill.serialize()
// skillSnapshot: { yourJobIfYouChooseToAcceptItIs, state, stateSchema, ... }

skill.unserialize(skillSnapshot)

unserialize restores state and skill options but does not reconnect callback handlers — those are defined in code. Re-attach any callbacks when recreating the skill.

API Reference

Bot methods

| Method | Description | |--------|-------------| | sendMessage(message, cb?) | Send a user message (string or { imageBase64, imageDescription }). Optional callback for each response. | | getIsDone() | Check if conversation is complete | | markAsDone() | Force conversation completion | | clearMessageHistory() | Drop all tracked messages | | updateState(partialState) | Update state and emit did-update-state | | setSkill(skill) | Swap the active skill | | serialize() | Snapshot of bot's current state, skill, and history | | unserialize(serialized) | Restore state from a previous serialize() snapshot |

Skill methods

| Method | Description | |--------|-------------| | updateState(partialState) | Update skill state | | getState() | Get current state | | setModel(model) | Change the model this skill uses | | serialize() | Snapshot of skill configuration and state | | unserialize(serialized) | Restore skill state from a previous serialize() snapshot |

Factory helpers

SprucebotLlmFactory also exposes:

• setBotInstance(bot) and getBotInstance() for storing a single bot instance.
• SprucebotLlmFactory.BotClass, .SkillClass, .FactoryClass overrides for dependency injection in tests.
• SprucebotLlmFactory.reset() to restore defaults.

Errors

SprucebotLlmError is exported for structured error handling:

import { SprucebotLlmError } from '@sprucelabs/sprucebot-llm'

try {
    await bot.sendMessage('hello')
} catch (err) {
    if (err instanceof SprucebotLlmError) {
        console.log(err.options?.code) // Error code
        console.log(err.friendlyMessage()) // Human-readable message
    }
}

Common error codes:

• NO_BOT_INSTANCE_SET
• INVALID_CALLBACK
• CALLBACK_ERROR

Testing utilities

These are exported from the package for unit tests:

| Utility | Description | |---------|-------------| | SpyLlmAdapter | Captures the last bot and options passed to the adapter | | SpyLllmBot | Records constructor options and exposes message history helpers | | MockLlmSkill | Assertion helpers for skill configuration and callbacks | | SpyOpenAiApi | Drop-in OpenAI client stub for adapter tests |

// Example: Using SpyOpenAiApi for tests
import { OpenAiAdapter, SpyOpenAiApi } from '@sprucelabs/sprucebot-llm'

OpenAiAdapter.OpenAI = SpyOpenAiApi
const adapter = OpenAiAdapter.Adapter('fake-key')

Development

Useful commands from package.json:

yarn test           # Run tests
yarn build.dev      # Development build
yarn build.dist     # Production build
yarn chat           # Interactive chat demo
yarn chat.images    # Chat with image support
yarn generate.samples