AI SDK Observational Memory

Observational Memory (OM) for AI SDK v6 in TypeScript.

This package implements a Mastra-inspired two-agent memory system:

• Observer compresses raw message history into dense observations
• Reflector condenses observations when they become too large

It is designed for ToolLoop/agent-loop usage with stable context and prompt-cache-friendly memory blocks.

Default runtime behavior mirrors Mastra OM turn-taking semantics:

• prompt uses memory prefix + unobserved message window
• observation/reflection run after each completed step in background
• synchronous observation only occurs under hard token pressure (observation.blockAfter)

Installation

pnpm add ai-sdk-observational-memory

Quick Start

import { createObservationalMemory } from "ai-sdk-observational-memory";
import { InMemoryObservationalMemoryStorage } from "ai-sdk-observational-memory/storage";

const om = createObservationalMemory({
  storage: new InMemoryObservationalMemoryStorage(),
  model: observerAndReflectorModel,
  observation: {
    messageTokens: 30_000,
    promptRecentMessages: 6,
    bufferTokens: 0.2,
    bufferActivation: 0.8,
    blockAfter: 1.2,
  },
  reflection: {
    observationTokens: 40_000,
    bufferActivation: 0.5,
    blockAfter: 1.2,
  },
});

ToolLoop Integration

import { createToolLoopMemoryHooks } from "ai-sdk-observational-memory/toolloop";

const hooks = createToolLoopMemoryHooks(om);

// before each step
const patch = await hooks.onBeforeStep({
  stepNumber,
  threadId,
  resourceId,
  messages,
});

// after each step
await hooks.onAfterStep({
  stepNumber,
  threadId,
  resourceId,
  messages,
  inputMessages,
  outputMessages,
});

AI SDK Middleware Integration

Use AI SDK language model middleware to enable OM for generateText, generateObject, and streamText.

import { generateText, openai } from "ai";
import {
  createObservationalMemory,
  withObservationalMemoryModel,
} from "ai-sdk-observational-memory";
import { InMemoryObservationalMemoryStorage } from "ai-sdk-observational-memory/storage";

const om = createObservationalMemory({
  storage: new InMemoryObservationalMemoryStorage(),
  model: observerAndReflectorModel,
});

const model = withObservationalMemoryModel({
  model: openai("gpt-4.1-mini"),
  om,
  context: {
    threadId: "thread-123",
    resourceId: "resource-123",
    stepNumber: 1,
  },
});

const result = await generateText({
  model,
  prompt: "Summarize the latest updates.",
});

You can also pass call context through providerOptions.observationalMemory per request:

const result = await generateText({
  model: withObservationalMemoryModel({ model: openai("gpt-4.1-mini"), om }),
  prompt: "Status update?",
  providerOptions: {
    observationalMemory: {
      threadId: "thread-123",
      resourceId: "resource-123",
      stepNumber: 1,
      currentDate: new Date().toISOString(),
    },
  },
});

Convenience wrappers are also available:

import {
  withObservationalMemoryGenerateText,
  withObservationalMemoryGenerateObject,
  withObservationalMemoryStreamText,
} from "ai-sdk-observational-memory";

Development TUI (Real Model via AI Gateway)

This repo includes a local dev TUI that uses a real model through AI Gateway and keeps Observational Memory state between turns.

export AI_GATEWAY_API_KEY=...
export DEV_TUI_MODEL=openai/gpt-4.1-mini
pnpm run dev:tui

Optional environment variables:

• DEV_TUI_OM_MODEL (defaults to DEV_TUI_MODEL)
• DEV_TUI_THREAD_ID (defaults to dev-thread)
• DEV_TUI_RESOURCE_ID (defaults to DEV_TUI_THREAD_ID)
• DEV_TUI_OBSERVATION_MESSAGE_TOKENS (default 900)
• DEV_TUI_REFLECTION_OBSERVATION_TOKENS (default 1800)

Commands inside the TUI:

• /memory show current active observations
• /clear clear memory for current thread/resource
• /help show command list
• /exit quit

Key Config

• observation.messageTokens: observe threshold for unobserved messages
• observation.promptRecentMessages: fallback count of recent non-system messages to keep if everything is already observed (default 6)
• observation.bufferTokens: async buffering interval (false disables buffering)
• observation.bufferActivation: percent of buffered observations to activate
• observation.blockAfter: hard token-pressure threshold that forces synchronous observation before actor call
• reflection.observationTokens: reflection threshold
• reflection.bufferActivation: async reflection trigger ratio
• reflection.blockAfter: force sync reflection threshold

Current Limitations

• In-memory storage is included; production adapters should implement ObservationalMemoryStorage.
• shareTokenBudget currently requires observation.bufferTokens: false.
• ToolLoop wrapper support is generic and may need a thin adapter for your loop runtime shape.

API

Main exports:

• createObservationalMemory
• ObservationalMemory
• createToolLoopMemoryHooks
• withObservationalMemory
• createObservationalMemoryMiddleware
• withObservationalMemoryModel
• withObservationalMemoryGenerateText
• withObservationalMemoryGenerateObject
• withObservationalMemoryStreamText
• TokenCounter
• Observer/reflector prompt + parser helpers

License

MIT