agui-framework

A TypeScript SDK for building AI agent-powered applications. agui-framework provides a complete toolkit for creating, orchestrating, and deploying LLM-based agents with multi-provider support, real-time streaming, state management, persistence, and the AG-UI protocol for frontend communication.

Features

• Agent class -- Central orchestrator for LLM interactions with run/stream/resume execution modes
• Multi-LLM providers -- OpenAI, Anthropic, Ollama, and Fireworks support with a common abstraction
• Real-time streaming -- AsyncGenerator-based streaming with event callbacks
• Event system -- Publish/subscribe EventBus with history, compaction, and piping
• Structured output -- JSON Schema-enforced responses via response_format
• State management -- Thread-isolated SharedState with versioning, diffing, merging, and conflict resolution; agents can autonomously read/write global shared state via built-in setState/getState/deleteState tools
• Long-term memory -- Optional RDF-based semantic store (Oxigraph) with self-managing remember/recall/forget tools
• AG-UI protocol -- Full SSE-based protocol encoding, validation, and event compaction
• MCP (Model Context Protocol) -- Connect to any MCP-compatible tool server via stdio or streamable HTTP; tools are auto-discovered and registered
• Multi-agent patterns -- Delegation, cyclic handoff, capability routing, and directed graph workflows
• Middleware pipeline -- Composable event interception and transformation
• Persistence -- Memory, Redis, and Postgres thread stores
• HTTP/WebSocket server -- Express route handlers, WebSocket agent communication, model catalog API
• Model catalog -- 44 models across 4 providers with pricing, context windows, and capabilities
• Cost & usage tracking -- Per-run token usage, cost calculation, cumulative thread cost, budget limits
• WebSocket client -- Full-duplex agent communication with run/stream/resume/capabilities
• React client hooks -- useStream, useThread, useInterrupts, useCoAgent, useWebSocket, useAgentState, useRunningAgents, useLiveState, and more
• State REST API -- Server endpoints for reading/writing agent shared state; AguiClient methods and React hooks included
• Type safety -- Full TypeScript with strict types across all modules

Installation

npm install agui-framework

Requires Node.js 18+ for native fetch and ReadableStream support.

Optional persistence dependencies:

npm install ioredis          # RedisThreadStore
npm install pg               # PostgresThreadStore

Optional long-term memory:

npm install oxigraph         # OxigraphSemanticStore

Quick Start

1. Basic Agent

import { Agent } from "agui-framework";

const agent = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "You are a helpful assistant.",
});

const response = await agent.run("What is the capital of France?");
console.log(response);

Set OPENAI_API_KEY in your environment.

2. Streaming Response

import { Agent } from "agui-framework";

const agent = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "Tell a detailed story.",
});

for await (const chunk of agent.stream("Tell me about space exploration")) {
  process.stdout.write(chunk);
}

3. Agent with Tools

import { Agent } from "agui-framework";
import type { ToolConfig } from "agui-framework";

const weatherTool: ToolConfig = {
  name: "get_weather",
  description: "Get current weather for a city",
  parameters: {
    type: "object",
    properties: { city: { type: "string", description: "City name" } },
    required: ["city"],
  },
  handler: async ({ city }) => ({ city, temperature: 22, conditions: "sunny" }),
};

const agent = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "Use the weather tool when asked about weather.",
  tools: [weatherTool],
});

const response = await agent.run("What is the weather in Paris?");
console.log(response);

4. Structured Output

import { Agent } from "agui-framework";

const agent = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "Extract structured data from text.",
  structuredOutput: true,
  outputSchema: {
    name: "extract_person",
    schema: {
      type: "object",
      properties: {
        name: { type: "string" },
        age: { type: "number" },
        occupation: { type: "string" },
      },
      required: ["name", "occupation"],
    },
  },
});

const response = await agent.run("Alice is a 30-year-old engineer.");
console.log(response); // JSON string: { "name": "Alice", "age": 30, "occupation": "engineer" }

5. Global Shared State

A SharedState instance can be passed to one or more agents. The agent automatically registers setState/getState/deleteState tools that read and write the shared state — no thread scoping required.

import { Agent, SharedState } from "agui-framework";

const shared = new SharedState({ theme: "dark", maxRetries: 3 });

const agent = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "You are a helpful assistant. Use setState to remember user preferences.",
  sharedState: shared,  // enables setState/getState/deleteState tools
});

const response = await agent.run("Remember my favorite color is blue.");
// The agent calls setState("favoriteColor", "blue") autonomously
console.log(shared.get("favoriteColor")); // "blue"

// Multiple agents or users can share the same state:
const agent2 = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "You know the user's preferences.",
  sharedState: shared,
});

const reply = await agent2.run("What is my favorite color?");
// The agent calls getState("favoriteColor") and responds "blue"

6. MCP (Model Context Protocol)

Connect any MCP-compatible tool server and use its tools automatically:

import { Agent } from "agui-framework";

const agent = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "You have access to MCP tools.",
  mcpServers: [
    // Local subprocess via stdio
    { transport: "stdio", command: "node", args: ["path/to/mcp-server.js"] },
    // Remote server via streamable HTTP
    { transport: "streamable-http", url: "https://mcp.example.com/tools" },
  ],
});

// MCP tools are auto-discovered and merged into the agent's tool list
const tools = agent.getTools(); // includes MCP tools + any configured tools

// Tools are available during run/stream execution
const response = await agent.run("Use the connected MCP tools.");

7. Multi-Provider Configuration

import { Agent } from "agui-framework";

// Anthropic
const agent = new Agent({
  model: "claude-3-5-sonnet-20240620",
  provider: "anthropic",
  instructions: "You are helpful.",
});

// Ollama (local)
const localAgent = new Agent({
  model: "llama3",
  provider: "ollama",
  instructions: "You are helpful.",
  baseUrl: "http://localhost:11434/v1",
});

// Fireworks
const fwAgent = new Agent({
  model: "accounts/fireworks/models/deepseek-v4-flash",
  provider: "fireworks",
  instructions: "You are helpful.",
});

8. Long-Term Memory

import { Agent, OxigraphSemanticStore, createLTMMiddleware } from "agui-framework";

const store = new OxigraphSemanticStore();
const agent = new Agent({
  model: "gpt-4o",
  provider: "openai",
  instructions: "You are a helpful assistant.",
});

agent.use(createLTMMiddleware(store));

// The agent autonomously calls remember/recall/forget tools
const reply = await agent.run(
  "Remember I prefer short answers.",
  { userId: "alice" },
);

9. Environment-Based Configuration

AGUI_PROVIDER=openai
AGUI_MODEL=gpt-4o
AGUI_INSTRUCTIONS=You are a helpful assistant.
AGUI_MAX_TOKENS=1024

import "dotenv/config";
import { Agent } from "agui-framework";

const agent = Agent.createFromEnv();
const reply = await agent.run("Tell me a joke");
console.log(reply);

Configuration

AgentConfig

| Option | Type | Default | Description | | ------------------------ | ------------------------- | ---------------- | --------------------------------------------------------------------------------------------- | | name | string | undefined | Display name for the agent | | model | string | required | LLM model identifier | | provider | ProviderType | required | openai, anthropic, ollama, fireworks | | instructions | string | required | System prompt | | tools | ToolConfig[] | [] | Tool definitions | | maxTokens | number | 1024 | Maximum output tokens | | temperature | number | 0.7 | Sampling temperature (0-2) | | topP | number | undefined | Nucleus sampling | | stream | boolean | false | Enable streaming by default | | apiKey | string | env var | API key for the provider | | baseUrl | string | provider default | Custom API endpoint | | store | ThreadStore | undefined | Persistence backend | | autoPersist | boolean | true | Auto-save to store after runs | | capabilities | string[] | [] | Custom capability flags | | maxIterations | number | 10 | Execution iteration limit | | maxExecutionTime | number | 30000 | Max wall-clock time in ms | | structuredOutput | boolean | false | JSON schema output support | | outputSchema | Record<string, unknown> | undefined | JSON Schema for enforced structured output | | supportedMimeTypes | string[] | ['text/plain'] | Output MIME types | | parallelCalls | boolean | false | Parallel tool calls | | clientProvidedTools | boolean | false | Client-injectable tools | | websocket | boolean | false | WebSocket transport | | codeExecution | boolean | false | Declares an externally isolated code-execution capability; no in-process executor is supplied | | sandboxed | boolean | false | Metadata for an externally sandboxed tool | | reasoningEncrypted | boolean | false | Requires reasoningEncryptionKey; uses AES-256-GCM | | reasoningEncryptionKey | string | undefined | Base64-encoded 32-byte AES-256-GCM key | | humanInterventions | boolean | false | Mid-run intervention | | humanFeedback | boolean | false | Human feedback (ratings) | | approveWithEdits | boolean | false | Approve with edits | | sharedState | SharedState | undefined | Global SharedState instance; registers setState/getState/deleteState tools on the agent | | mcpServers | MCPServerConfig[] | undefined | MCP server configurations; connects on init and registers auto-discovered tools |

Environment Variables

| Provider | Variable | AGUI prefix alternative | | --------- | ------------------- | ----------------------------------------------------------------------------------------- | | OpenAI | OPENAI_API_KEY | AGUI_PROVIDER, AGUI_MODEL, AGUI_INSTRUCTIONS, AGUI_MAX_TOKENS, AGUI_TEMPERATURE | | Anthropic | ANTHROPIC_API_KEY | | | Ollama | OLLAMA_API_KEY | | | Fireworks | FIREWORKS_API_KEY | |

Architecture Overview

agui-framework
  Agent            -- Orchestrates LLM calls, tools, events, state, persistence
  EventBus         -- In-process pub/sub with history and compaction
  StateManager     -- Thread-isolated SharedState instances
  SharedState      -- Global key-value store passed to agents via sharedState config; accessed via built-in setState/getState/deleteState tools
  MCPClientManager -- Connects to MCP servers, discovers tools, and registers them as ToolConfig entries
  ProtocolEncoder  -- Event serialization and SSE encoding
  ProtocolValidator-- Input and event validation
  BaseLLMProvider  -- Abstract LLM provider (OpenAI, Anthropic, Ollama, Fireworks)
  MiddlewareChain  -- Composable middleware pipeline
  MultiAgentManager-- Agent delegation, cyclic handoff, capability routing, graph execution
  AgentGraph       -- Directed graph of agent nodes
  ThreadStore      -- Persistence interface (Memory, Redis, Postgres)
  SemanticStore    -- RDF-based long-term memory interface (Oxigraph)
  LTMiddleware     -- Self-managing remember/recall/forget middleware
  AguiClient       -- HTTP client for remote agent execution
  React hooks      -- useStream, useThread, useInterrupts, useCoAgent

Documentation

| Document | Description | | ----------------------------------------------------------------- | ------------------------------------------------- | | Getting Started | Installation, prerequisites, first agent | | Agents | Agent class deep dive: config, tools, execution | | Events | EventBus API, event types, subscriptions | | State Management | SharedState, StateManager, thread isolation, global agent state tools | | Protocol | ProtocolEncoder, SSE, validation | | Providers | Provider architecture, custom providers | | Tools | ToolConfig, handlers, interrupts, delegation | | Architecture | Module relationships, data flow, extension points | | API Reference | Complete API reference organized by module | | Future Integrations | Draft proposals for router, meta events, GAI | | Examples | Multi-agent, graph, persistence, React, Express | | AG-UI Agents | AG-UI protocol agent implementation | | AG-UI Events | AG-UI event format and types implementation |

API Overview

Agent

class Agent {
  constructor(config: AgentConfig);
  run(prompt: string, context?: Partial<RunContext>): Promise<string>;
  stream(
    prompt: string,
    context?: Partial<RunContext>,
    options?: StreamingOptions,
  ): AsyncGenerator<string>;
  resume(
    interruptId: string,
    payload?: unknown,
    status?: "resolved" | "cancelled",
  ): Promise<string>;
  addTool(tool: ToolConfig): this;
  addCapability(capability: string): this;
  use(...middlewares: MiddlewareFunction[]): this;
  delegate(
    subAgent: Agent,
    prompt: string,
    config?: DelegationConfig,
  ): Promise<string>;
  createDelegationTool(
    toolName: string,
    description: string,
    subAgent: Agent,
  ): ToolConfig;
  createHandoffTool(
    toolName: string,
    description: string,
    targetAgent: Agent,
  ): ToolConfig;
  clone(): Agent;
  getCapabilities(
    peerDescriptors?: Map<string, AgentDescriptor>,
  ): AgentCapabilities;
  loadThread(threadId: string): Promise<void>;
  saveThread(threadId: string): Promise<void>;
  toJSON(): string;
  fromJSON(json: string): this;
  static create(config: AgentConfig): Agent;
  static createFromEnv(): Agent;
}

interface AgentConfig {
  name?: string;
  model: string;
  provider: ProviderType;
  instructions: string;
  tools?: ToolConfig[];
  maxTokens?: number;
  temperature?: number;
  topP?: number;
  stream?: boolean;
  modelSettings?: Record<string, unknown>;
  capabilities?: string[];
  apiKey?: string;
  baseUrl?: string;
  store?: ThreadStore;
  autoPersist?: boolean;
  maxIterations?: number;
  maxExecutionTime?: number;
  structuredOutput?: boolean;
  supportedMimeTypes?: string[];
  parallelCalls?: boolean;
  clientProvidedTools?: boolean;
  websocket?: boolean;
  codeExecution?: boolean;
  sandboxed?: boolean;
  reasoningEncrypted?: boolean;
  humanInterventions?: boolean;
  humanFeedback?: boolean;
  approveWithEdits?: boolean;
  sharedState?: SharedState;
  mcpServers?: MCPServerConfig[];
  multimodalInput?: {
    image?: boolean;
    audio?: boolean;
    video?: boolean;
    pdf?: boolean;
    file?: boolean;
  };
  multimodalOutput?: { image?: boolean; audio?: boolean };
}

EventBus

class EventBus {
  constructor(maxHistory?: number);
  emit(event: AgentEvent): void;
  on(type: EventType | "*", listener: EventListener): () => void;
  once(type: EventType | "*", listener: EventListener): void;
  clear(): void;
  getHistory(): BaseEvent[];
  compact(): void;
  pipe(transform: (event: AgentEvent) => AgentEvent | null): EventBus;
  toJSON(): string;
  fromJSON(json: string): void;
}

StateManager / SharedState

class StateManager {
  constructor(defaultState?: SharedState);
  getOrCreateState(threadId: string): SharedState;
  updateState(threadId: string, updates: Partial<StateData>): SharedState;
  deleteState(threadId: string): void;
  hasState(threadId: string): boolean;
  getAllThreads(): string[];
  clearAll(): void;
  exportState(threadId: string): StateSnapshot | null;
  importState(threadId: string, snapshot: StateSnapshot): SharedState;
  subscribe(subscription: StateSubscription): () => void;
}

class SharedState {
  constructor(
    initialData?: StateData,
    options?: StateOptions,
    threadId?: string,
  );
  get<T>(key: string, defaultValue?: T): T;
  set(key: string, value: unknown): this;
  update(updates: Partial<StateData>): this;
  delete(key: string): this;
  has(key: string): boolean;
  clear(): void;
  takeSnapshot(label?: string): void;
  getSnapshot(): StateSnapshot;
  getHistory(limit?: number): StateSnapshot[];
  diff(other: SharedState): StateDiff;
  merge(
    other: SharedState,
    strategy?: StateMergeStrategy,
    conflictHandler?: Function,
  ): StateData;
  computePatch(): JsonPatchOperation[];
  getVersion(): string;
  subscribe(subscription: StateSubscription): () => void;
  toJSON(): string;
  fromJSON(json: string): this;
}

Provider Types

type ProviderType = "openai" | "anthropic" | "ollama" | "fireworks";

abstract class BaseLLMProvider {
  readonly type: ProviderType;
  config: ProviderConfig;
  chatCompletion(
    request: ChatCompletionRequest,
  ): Promise<ChatCompletionResponse>;
  streamChatCompletion(
    request: ChatCompletionRequest,
  ): AsyncGenerator<StreamChunk>;
  prepareMessages(
    systemPrompt: string,
    messages: Array<{ role: string; content: string }>,
  ): ChatMessage[];
}

ThreadStore Interface

interface ThreadStore {
  connect(): Promise<void>;
  disconnect(): Promise<void>;
  createThread(
    threadId: string,
    metadata?: Record<string, unknown>,
    agentId?: string,
    ownerId?: string,
  ): Promise<ThreadData>;
  getThread(threadId: string): Promise<ThreadData | null>;
  listThreads(limit?: number, offset?: number, agentId?: string, userId?: string): Promise<ThreadData[]>;
  deleteThread(threadId: string): Promise<void>;
  appendMessages(threadId: string, messages: Message[]): Promise<void>;
  getMessages(
    threadId: string,
    limit?: number,
    offset?: number,
  ): Promise<Message[]>;
  saveState(threadId: string, state: Record<string, unknown>): Promise<void>;
  getState(threadId: string): Promise<Record<string, unknown> | null>;
  saveRun(runId: string, threadId: string, data: RunData): Promise<void>;
  getRun(runId: string): Promise<RunData | null>;
  searchMessages(
    threadId: string,
    query: string,
    limit?: number,
  ): Promise<Message[]>;
}

Implementations: MemoryThreadStore, RedisThreadStore, PostgresThreadStore.

SemanticStore (Long-Term Memory)

import { OxigraphSemanticStore } from "agui-framework";

const store = new OxigraphSemanticStore();
await store.remember("alice", [
  { subject: "alice", predicate: "prefersModel", object: "gpt-4o", timestamp: Date.now() },
]);
const facts = await store.recall("alice"); // returns all facts for alice
await store.forget("alice", "prefersModel"); // delete specific fact

The createLTMMiddleware wraps this store into the agent's execution pipeline, injecting remember/recall/forget tools so the agent manages its own memory. Each user's facts live in isolated named RDF graphs.

Model Catalog & Cost Tracking

import {
  getModel,
  calculateCost,
  formatCost,
  modelCatalog,
} from "agui-framework";

// Look up a model's pricing and capabilities
const model = getModel("gpt-5.6-luna");
console.log(model?.pricing); // { input: 1.00, output: 6.00, cachedInput: 0.10 }

// Calculate cost from token usage
const cost = calculateCost("deepseek-v4-flash", {
  promptTokens: 5000,
  completionTokens: 1200,
  totalTokens: 6200,
});
console.log(formatCost(cost!.totalCost)); // "$0.0010"

// Check context window limits
const { exceeds, limit } = exceedsContextWindow("llama-3.2-3b", 200_000);
console.log(exceeds); // true, limit is 128000

// Query from server
const client = new AguiClient("http://localhost:4124");
const models = await client.models();
const gptModel = await client.model("gpt-5.6-terra");

Costs are automatically tracked per run and accumulated per thread when a store is configured:

GET /api/threads              → includes totalCost, runCount, lastModelId
GET /api/threads/:id/runs     → includes usage + cost per run

React Hooks

// From 'agui-framework/client/react'
function useStream(): { start; stop; isLoading; error; result };
function useThread(options: UseThreadOptions): {
  threads;
  messages;
  loadMessages;
  createThread;
  currentThreadId;
};
function useInterrupts(): { interrupts; handleInterrupt; resolve; clear };
function useCoAgent(options: UseCoAgentOptions): UseCoAgentReturn;
function useCoAction(
  toolDef: ToolConfig & { handler: Function },
): UseCoActionReturn;
function useAgentState(
  agentId: string,
  baseUrl: string,
): { state; setState; deleteState; loading; error; refetch };
function useCapabilities(
  agentId: string,
  baseUrl: string,
): { caps; loading; error; refetch };
function useAgent(
  agentId: string,
  baseUrl: string,
): { meta; loading; error; refetch };
function useWebSocket(
  baseUrl: string,
  agentId: string,
): { connected; caps; connect; disconnect; on; off; run; resume };

Event Types

| Category | Events | | -------------- | ------------------------------------------------------------------------------------------------- | | Run lifecycle | RUN_STARTED, RUN_FINISHED, RUN_ERROR | | Steps | STEP_STARTED, STEP_FINISHED | | Text messages | TEXT_MESSAGE_START, TEXT_MESSAGE_CONTENT, TEXT_MESSAGE_END | | Tool calls | TOOL_CALL_START, TOOL_CALL_ARGS, TOOL_CALL_END, TOOL_CALL_RESULT | | State | STATE_SNAPSHOT, STATE_DELTA, MESSAGES_SNAPSHOT | | Activity | ACTIVITY_SNAPSHOT, ACTIVITY_DELTA | | Reasoning | REASONING_START, REASONING_END, REASONING_MESSAGE_*, REASONING_ENCRYPTED_VALUE | | Multi-agent | AGENT_DELEGATION_START, AGENT_DELEGATION_END, AGENT_HANDOFF_REQUEST, AGENT_HANDOFF_RESULT | | Human-in-loop | HUMAN_INTERVENTION_REQUEST, HUMAN_INTERVENTION_RESULT, HUMAN_FEEDBACK | | Code execution | CODE_EXECUTION_START, CODE_EXECUTION_RESULT | | Memory | MEMORY_SUMMARY | | Usage & cost | USAGE_UPDATE | | Custom | RAW, CUSTOM |

License

MIT