@sisu-ai/adapter-ollama

v12.0.0

Published

2 months ago

Connect Sisu to local Ollama models with native tool support and streaming.

0High
0Medium
0Low

lejahmie

sisu ai ai-agent agentic adapter ollama

@sisu-ai/adapter-ollama

Connect Sisu to local Ollama models with native tool support and streaming.

Setup

npm i @sisu-ai/adapter-ollama

Start Ollama locally: ollama serve
Pull a tools-capable model: ollama pull llama3.1:latest
Base URL env: BASE_URL

Transport and compatibility

The adapter now uses the official ollama JavaScript client for chat transport.
Public Sisu behavior remains stable: normalized messages/tool-calls, streaming events, and image URL-to-base64 preprocessing.
GenerateOptions.toolChoice semantics are normalized at the adapter layer:
- auto / required keeps all declared tools available to the model
- none omits tools for that call
- named tool choice narrows the sent tool list to that tool
Cancellation is propagated for request execution and image preprocessing fetches.

Usage

import { ollamaAdapter, ollamaEmbeddings } from '@sisu-ai/adapter-ollama';

const model = ollamaAdapter({ model: 'llama3.1' });
// or with custom base URL: { baseUrl: 'http://localhost:11435' }

const embeddings = ollamaEmbeddings({ model: 'embeddinggemma' });
const vectors = await embeddings.embed(['first text', 'second text']);

Images (Vision)

Accepts multi-part content arrays with type: 'text' | 'image_url' and convenience fields like images/image_url.
The adapter maps these to Ollama's expected shape by sending content as a string and images as a string array on the message.
If an image value is an http(s) URL, the adapter fetches it and inlines it as base64 automatically. Data URLs are supported; raw base64 strings pass through.

Content parts (adapter maps to images[] under the hood and auto-fetches URLs):

const messages: any[] = [
  { role: 'user', content: [
    { type: 'text', text: 'What is in this image?' },
    { type: 'image_url', image_url: { url: 'https://example.com/pic.jpg' } },
  ] }
];
const res = await model.generate(messages, { toolChoice: 'none' });

Convenience shape:

const messages: any[] = [
  { role: 'user', content: 'Describe the image.', images: ['https://example.com/pic.jpg'] },
];
const res = await model.generate(messages, { toolChoice: 'none' });

Normalizing Ollama API

Providers such as OpenAI vision models accepts image_url parts with url pointing to a remote image; the provider dereferences the URL.
Ollama expects each message to include images: string[] of base64-encoded image data; it does not dereference remote URLs.
This adapter keeps the authoring experience consistent by accepting OpenAI-style parts and convenience URLs, and performs URL→base64 conversion for you.

Accepted image formats

Base64 string: images: ["<base64>"] (preferred/default for Ollama)
Data URL: images: ["data:image/png;base64,<base64>"] or in parts via { type: 'image_url', image_url: { url: 'data:...' } }
Remote URL (convenience): { type: 'image_url', image_url: { url: 'https://...' } } or images: ['https://...'] — adapter fetches and inlines as base64.

Note: URL fetching happens from your runtime. If your environment blocks outbound HTTP, either provide base64 directly or host images where your runtime can reach them.

Tools

Define tools as small, named functions with a zod schema.
Register them on your agent and add the tool-calling middleware — the adapter handles the wire format to/from Ollama.
Under the hood, the adapter sends your tool schemas to the model, maps model “function calls” back to your handlers, and includes tool results for follow‑up turns.

Quick start with tools

import { Agent, InMemoryKV, NullStream, SimpleTools, createConsoleLogger, type Ctx, type Tool } from '@sisu-ai/core';
import { registerTools } from '@sisu-ai/mw-register-tools';
import { toolCalling } from '@sisu-ai/mw-tool-calling';
import { z } from 'zod';
import { ollamaAdapter } from '@sisu-ai/adapter-ollama';

const sum: Tool<{ a: number; b: number }> = {
  name: 'sum',
  description: 'Add two numbers',
  schema: z.object({ a: z.number(), b: z.number() }),
  handler: async ({ a, b }) => ({ result: a + b }),
};

const model = ollamaAdapter({ model: 'llama3.1' });
const ctx: Ctx = {
  input: 'Use the sum tool to add 3 and 7, then explain.',
  messages: [{ role: 'system', content: 'You are helpful.' }],
  model,
  tools: new SimpleTools(),
  memory: new InMemoryKV(),
  stream: new NullStream(),
  state: {},
  signal: new AbortController().signal,
  log: createConsoleLogger(),
};

const app = new Agent()
  .use(registerTools([sum])) // make tools available
  .use(toolCalling);         // let the model pick tools, run them, and finalize

await app.handler()(ctx);

Notes

Tool choice forcing is model-dependent; current loop asks for tools on first turn and plain completion on second.
Streaming is supported and mapped to Sisu token + final assistant_message events.
Env: BASE_URL overrides the base URL (or pass baseUrl in code). Examples may also support a CLI flag --base-url to override env.

Community & Support

Discover what you can do through examples or documentation. Check it out at https://github.com/finger-gun/sisu. Example projects live under examples/ in the repo.