@ai-sdk-tool/harness

A lightweight, model-agnostic agent harness built on the Vercel AI SDK. Provides the core loop, message history management, session lifecycle, skills loading, TODO continuation, command registry, and tool orchestration primitives for building AI agents.

Installation

pnpm add @ai-sdk-tool/harness
# or
npm install @ai-sdk-tool/harness

Peer dependencies:

pnpm add ai zod

Quick Start

import { defineAgent, createAgentRuntime } from "@ai-sdk-tool/harness/runtime";
import { FileSnapshotStore } from "@ai-sdk-tool/harness/sessions";
import { runAgentSessionTUI } from "@ai-sdk-tool/tui/session";

const assistant = defineAgent({
  name: "assistant",
  agent: {
    model,
    instructions: "You are a helpful assistant.",
    tools,
  },
  history: {
    compaction: { enabled: true, contextLimit: 100_000 },
  },
});

const runtime = await createAgentRuntime({
  name: "my-assistant",
  agents: [assistant],
  persistence: { snapshotStore: new FileSnapshotStore(".my-assistant") },
});

const session = await runtime.openSession();
await runAgentSessionTUI(session);

See Low-level API for direct createAgent / runAgentLoop / CheckpointHistory usage.

Subpath Imports

// High-level runtime API (recommended starting point)
import { defineAgent, createAgentRuntime } from "@ai-sdk-tool/harness/runtime";

// Persistence
import { FileSnapshotStore, InMemorySnapshotStore } from "@ai-sdk-tool/harness/sessions";

// Compaction utilities
import { createModelSummarizer, CompactionCircuitBreaker } from "@ai-sdk-tool/harness/compaction";

// Memory tracking
import { SessionMemoryTracker, BackgroundMemoryExtractor } from "@ai-sdk-tool/harness/memory";

Low-level API

import { createAgent, runAgentLoop, CheckpointHistory } from "@ai-sdk-tool/harness";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";
import { tool } from "ai";

// 1. Create an agent with a model and tools
const agent = await createAgent({
  model: openai("gpt-4o"),
  instructions: "You are a helpful assistant.",
  tools: {
    get_time: tool({
      description: "Get the current time",
      parameters: z.object({}),
      execute: async () => new Date().toISOString(),
    }),
  },
});

// 2. Run the agent loop
const result = await runAgentLoop({
  agent,
  messages: [{ role: "user", content: "What time is it?" }],
  onToolCall: (call, ctx) => {
    console.log(`[${ctx.iteration}] Tool call: ${call.toolName}`);
  },
  onStepComplete: (step) => {
    console.log(`Step ${step.iteration} done (${step.finishReason})`);
  },
});

console.log(`Finished after ${result.iterations} iterations`);

API Reference

createAgent(config)

Creates an Agent instance that wraps a Vercel AI SDK streamText call.

import { createAgent } from "@ai-sdk-tool/harness";

const agent = await createAgent({
  model,                        // LanguageModel — required
  instructions,                 // string | (() => Promise<string>) — system prompt
  tools,                        // ToolSet — tool definitions
  maxStepsPerTurn,              // number — max tool-call steps per stream (default: 1)
  extraStopConditions,          // StopCondition[] — additional independent stop triggers
  experimental_repairToolCall,  // repair callback for malformed tool calls
});

Returns: Agent — an object with config and stream(opts) method.

runAgentLoop(options)

Runs the agent in a loop until a stop condition is met or maxIterations is reached.

import { runAgentLoop } from "@ai-sdk-tool/harness";

const result = await runAgentLoop({
  agent,           // Agent — required
  messages,        // ModelMessage[] — initial conversation history
  maxIterations,   // number — max loop iterations (default: unlimited)
  abortSignal,     // AbortSignal — for cancellation

  // Hooks
  onPrepareStep,   // (context) => partial AgentStreamOptions override, applied before onBeforeTurn
  onBeforeTurn,    // (context) => partial AgentStreamOptions override
  onInterrupt,     // ({ iteration, reason }, context) => void | Promise<void>
  shouldContinue,  // (finishReason, context) => boolean — custom continuation logic
  onToolLifecycle, // (lifecycle, context) => void | Promise<void>
  onToolCall,      // (call, context) => void | Promise<void>
  onStepComplete,  // (step) => void | Promise<void>
  onError,         // (error, context) => void | Promise<void> | { shouldContinue?, recovery? }
});

Returns: RunAgentLoopResult

interface RunAgentLoopResult {
  messages: ModelMessage[];       // Full conversation history after loop
  iterations: number;             // Number of iterations completed
  finishReason: AgentFinishReason; // Final finish reason
}

CheckpointHistory

Manages conversation history with configurable limits, compaction, and automatic cleanup of invalid message sequences.

import { CheckpointHistory } from "@ai-sdk-tool/harness";

const history = new CheckpointHistory({
  compaction: {
    enabled: true,
    summarizeFn: async (messages) => "Summary of earlier conversation...",
    speculativeStartRatio: 0.8,
  },
});

Key behaviors:

• addUserMessage() and addModelMessages() manage history state
• getMessagesForLLM() returns summary-prefixed messages suitable for the next model call
• prepareSpeculativeCompaction(), applyPreparedCompaction(), and compact() are the supported compaction entrypoints
• invalid tool-call/tool-result sequences are cleaned up automatically during trimming and compaction

SessionManager

Manages a UUID-based session ID lifecycle. Useful for stamping events and file paths with a consistent identifier.

import { SessionManager } from "@ai-sdk-tool/harness";

const session = new SessionManager("my-agent"); // optional prefix, default: "session"

const sessionId = session.initialize(); // => "my-agent-<uuid>"
console.log(session.getId());           // => "my-agent-<uuid>"
console.log(session.isActive());        // => true

Methods:

| Method | Returns | Description | |--------|---------|-------------| | initialize() | string | Generates and stores a new session ID | | getId() | string | Returns the current session ID; throws if not initialized | | isActive() | boolean | Returns true if initialize() has been called |

SkillsEngine

Discovers and loads skills from up to five directories: bundled, global skills, global commands, project skills, and project commands.

import { SkillsEngine, type SkillsConfig } from "@ai-sdk-tool/harness";

const config: SkillsConfig = {
  bundledDir: "./skills",           // Bundled skills shipped with the agent
  globalSkillsDir: "~/.agent/skills",
  globalCommandsDir: "~/.agent/commands",
  projectSkillsDir: ".agent/skills",
  projectCommandsDir: ".agent/commands",
};

const engine = new SkillsEngine(config);
const skills = await engine.loadSkills(); // SkillInfo[]

// Get content of a specific skill
const content = await engine.getSkillContent("my-skill");

SkillInfo fields:

| Field | Type | Description | |-------|------|-------------| | id | string | Unique skill identifier | | name | string | Display name | | description | string | Short description for autocomplete | | path | string | Absolute path to the skill file | | format | "legacy" \| "v2" \| "command" | Skill file format | | source | "bundled" \| "global" \| "project" \| "global-command" \| "project-command" | Where the skill was found | | argumentHint | string? | Hint shown in autocomplete for skills that take arguments |

TodoContinuation

Reads a todo JSON file and generates reminder messages for incomplete tasks. Used with runHeadless to keep the agent running until all TODOs are done.

import { TodoContinuation, type TodoConfig } from "@ai-sdk-tool/harness";

const todo = new TodoContinuation({
  todoDir: ".sisyphus/todos",   // Directory containing todo JSON files
  sessionId: "session-abc123",  // Used to locate the correct todo file
  promptTemplate: (todos) => `You have ${todos.length} tasks remaining.`,
  userMessageTemplate: (todos) => `Continue with: ${todos[0].content}`,
});

const reminder = await todo.getReminder();
// => { hasReminder: true, message: "..." }
// or { hasReminder: false, message: null }

TodoItem fields:

| Field | Type | Description | |-------|------|-------------| | id | string | Unique task ID | | content | string | Task description | | status | "pending" \| "in_progress" \| "completed" \| "cancelled" | Current status | | priority | "high" \| "medium" \| "low" | Task priority | | description | string? | Optional longer description |

Command Registry

A global registry for slash commands. Commands are registered once and available to both TUI and headless runtimes.

import {
  registerCommand,
  executeCommand,
  getCommands,
  isCommand,
  parseCommand,
  configureCommandRegistry,
  createHelpCommand,
  resolveRegisteredCommandName,
  isSkillCommandResult,
} from "@ai-sdk-tool/harness";

// Register a command
registerCommand({
  name: "model",
  description: "Switch the active model",
  aliases: ["m"],
  execute: async ({ args }) => {
    const newModel = args[0];
    return { success: true, message: `Switched to ${newModel}` };
  },
});

// Check if input is a command
if (isCommand("/model gpt-4o")) {
  const result = await executeCommand("/model gpt-4o");
  console.log(result?.message);
}

// Configure skill loading for /skill-name commands
configureCommandRegistry({
  skillLoader: async (name) => {
    const content = await loadSkillFile(name);
    return content ? { content, id: name } : null;
  },
});

Functions:

| Function | Description | |----------|-------------| | registerCommand(command) | Adds a command to the global registry | | executeCommand(input) | Parses and executes a command string | | getCommands() | Returns the full Map<string, Command> | | isCommand(input) | Returns true if input starts with / | | parseCommand(input) | Parses "/name arg1 arg2" into { name, args } | | configureCommandRegistry(config) | Sets the skill loader for skill-based commands | | createHelpCommand(getCommands) | Creates a /help command listing all registered commands | | resolveRegisteredCommandName(name) | Resolves an alias to its canonical command name | | isSkillCommandResult(result) | Type guard for SkillCommandResult |

buildMiddlewareChain(config)

Builds a middleware chain for wrapping language model calls. Useful for logging, caching, or modifying requests/responses.

import { buildMiddlewareChain, type MiddlewareConfig } from "@ai-sdk-tool/harness";
import { wrapLanguageModel } from "ai";

const middlewares = buildMiddlewareChain({
  middlewares: [loggingMiddleware, cachingMiddleware],
});

const wrappedModel = wrapLanguageModel({
  model: openai("gpt-4o"),
  middleware: middlewares[0], // or compose them
});

createAgentPaths(options)

Creates a consistent set of filesystem paths for agent configuration and TODO storage.

import { createAgentPaths } from "@ai-sdk-tool/harness";

const paths = createAgentPaths({
  configDirName: ".my-agent",
  todoDirName: "todos",
  todoBaseDir: "/tmp",  // optional, defaults to os.tmpdir()
});

// paths.configDir => ".my-agent"
// paths.todoDir   => "/tmp/todos"

shouldContinueManualToolLoop(finishReason, context)

The default continuation predicate used by runAgentLoop. Returns true when finishReason is "tool-calls".

import { shouldContinueManualToolLoop } from "@ai-sdk-tool/harness";

// Use as custom shouldContinue with additional logic
const result = await runAgentLoop({
  agent,
  messages,
  shouldContinue: (reason, ctx) => {
    if (ctx.iteration >= 10) return false; // Custom limit
    return shouldContinueManualToolLoop(reason, ctx);
  },
});

normalizeFinishReason(reason)

Normalizes provider-specific finish reason strings to a canonical AgentFinishReason.

import { normalizeFinishReason } from "@ai-sdk-tool/harness";

const normalized = normalizeFinishReason("tool_calls"); // => "tool-calls"

Compaction prompts

Built-in summarization prompts for CheckpointHistory compaction.

import {
  createModelSummarizer,
  DEFAULT_SUMMARIZATION_PROMPT,
  ITERATIVE_SUMMARIZATION_PROMPT,
} from "@ai-sdk-tool/harness";

const summarize = createModelSummarizer({
  model: openai("gpt-4o-mini"),
  prompt: DEFAULT_SUMMARIZATION_PROMPT,
});

const history = new CheckpointHistory({
  compaction: {
    enabled: true,
    speculativeStartRatio: 0.8,
    summarizeFn: summarize,
  },
});

Types

import type {
  Agent,
  AgentConfig,
  AgentStreamOptions,
  AgentStreamResult,
  AgentFinishReason,
  LoopContinueContext,
  LoopStepInfo,
  LoopHooks,
  RunAgentLoopOptions,
  RunAgentLoopResult,
  // Re-exported from Vercel AI SDK:
  LanguageModel,
  ModelMessage,
  Tool,
  ToolCallPart,
  ToolSet,
  // CheckpointHistory types:
  CompactionConfig,
  CompactionSummary,
  Message,
  CheckpointHistoryOptions,
  // Session:
  // (SessionManager is a class, not a type)
  // Skills:
  SkillInfo,
  SkillsConfig,
  // TODO:
  TodoConfig,
  TodoItem,
  // Commands:
  Command,
  CommandContext,
  CommandRegistryConfig,
  CommandResult,
  SkillCommandResult,
  // Middleware:
  MiddlewareConfig,
  // Paths:
  AgentPaths,
  AgentPathsOptions,
  // Tool pruning:
  PruneResult,
  PruningConfig,
} from "@ai-sdk-tool/harness";

Advanced Usage

Custom tool-call repair

const agent = await createAgent({
  model,
  experimental_repairToolCall: async ({ toolCall, error, messages, system }) => {
    // Return repaired tool call arguments, or null to skip repair
    console.warn(`Repairing tool call: ${toolCall.toolName}`, error);
    return null;
  },
});

Compacting long conversations

const history = new CheckpointHistory({
  compaction: {
    enabled: true,
    speculativeStartRatio: 0.8,
    summarizeFn: async (messages) => {
      // Use your model to summarize
      const summary = await generateSummary(messages);
      return summary;
    },
  },
});

Abort signal for cancellation

const controller = new AbortController();

// Cancel after 30 seconds
setTimeout(() => controller.abort(), 30_000);

const result = await runAgentLoop({
  agent,
  messages,
  abortSignal: controller.signal,
});

Full session setup

import {
  createAgent,
  CheckpointHistory,
  SessionManager,
  SkillsEngine,
  TodoContinuation,
  registerCommand,
  createHelpCommand,
  getCommands,
  createAgentPaths,
} from "@ai-sdk-tool/harness";

const paths = createAgentPaths({
  configDirName: ".my-agent",
  todoDirName: "todos",
});

const session = new SessionManager("my-agent");
const sessionId = session.initialize();

const history = new CheckpointHistory({});

const skillsEngine = new SkillsEngine({
  bundledDir: "./skills",
  projectSkillsDir: ".my-agent/skills",
});
const skills = await skillsEngine.loadSkills();

const todo = new TodoContinuation({
  todoDir: paths.todoDir,
  sessionId,
});

registerCommand(createHelpCommand(getCommands));

const agent = await createAgent({ model, instructions: "..." });

License

MIT