reasoning-bank

A framework-agnostic TypeScript implementation of ReasoningBank (Ouyang et al., ICLR 2026): agents that distill lessons from past trajectories and retrieve them on new tasks. Includes MaTTS (memory-aware test-time scaling).

What this is

ReasoningBank is a memory mechanism for LLM agents. After each task, it judges the trajectory, distills generalizable reasoning strategies from both successes and failures, and indexes them. On future tasks it retrieves the relevant strategies and injects them into the agent's context.

This package is a clean, framework-agnostic TypeScript implementation. It works with any agent loop, the raw Anthropic or OpenAI SDK, or your own ReAct loop. It does not depend on any agent framework.

The reference implementation from the paper is welded to specific benchmark harnesses. This one is a standalone library you can attach to anything. A sister Python package exists with the same API shape.

Install

npm install reasoning-bank

Node 18+. ESM and CJS builds are both shipped.

How it plugs into any agent loop

Three calls:

import { ReasoningBank, MiniLMEmbedder, type Turn } from "reasoning-bank";

const myLLM = async (prompt: string, opts?: { system?: string }) => {
  // wrap your provider however you like — must return a string
  return await callYourProvider(prompt, opts?.system);
};

const bank = new ReasoningBank({
  llm: myLLM,
  embedder: new MiniLMEmbedder(),
  scope: "my-project",
});

// 1. retrieve relevant past lessons before your agent runs
const memories = await bank.retrieve(task);
const systemBlock = memories.length ? bank.formatAsSystemBlock(memories) : undefined;

// 2. run your agent however you want, optionally prepending systemBlock
const answer = await myAgent(task, systemBlock);

// 3. ingest the trajectory so the bank learns from it
const trajectory: Turn[] = [
  { role: "user", content: task },
  { role: "assistant", content: answer },
];
await bank.ingestTrajectory(trajectory, task);

The bank accepts any trajectory shape (its own Turn type or plain dicts with role/content) and any async LLM callable. Pluggable embedder (MiniLM via @xenova/transformers by default — runs locally in pure JS) and store (in-memory by default; bring your own for persistence).

MaTTS

Memory-aware test-time scaling: run k rollouts in parallel, contrast them, distill higher-quality memories.

import { mattsRun } from "reasoning-bank";

const rollout = async () => {
  const answer = await myAgent(task);
  return [
    { role: "user", content: task },
    { role: "assistant", content: answer },
  ];
};

const { trajectories, memories } = await mattsRun(rollout, task, bank, { k: 3 });

Does it actually work?

This is the honest part.

The machinery works end-to-end: it judges trajectories, distills sensible transferable lessons, retrieves them, and injects them, with no framework dependency. That is verified.

Whether it produces a measurable performance lift is a separate question, and the answer depends heavily on the task distribution and the model. The Python sibling of this package was used to run four controlled experiments to find out, including a SWE-bench-lite harness with a no-retry control, a naive-retry control, a per-instance bank, and a persistent cross-instance bank, scored by the official SWE-bench Docker evaluator.

Headline findings (Haiku 4.5, full methodology and data in the Python repo's experiments/ directory):

• On task suites where the base model is already at its capability ceiling, the bank cannot help, because there is no failure to learn from. Two early experiments hit this.
• On SWE-bench-lite (a real spread of difficulty, ~50% baseline), the persistent cross-instance bank produced no measurable lift over a no-retry baseline. The cross-task transfer hypothesis was not supported in this setup (n=45 clean cells on the persistent arm; late instances did not outperform early ones).
• The one consistently positive observation was defensive: naive retry (re-prompting with the raw error) hurt performance, and structured-reflection retry recovered it. The value was in how a failure is framed to the model, not in accumulating a memory bank.

In short: the implementation is faithful and the machinery is sound, but I did not find a regime in these experiments where cross-task memory accumulation produced net positive value. Negative results are results.

Design

• Framework-agnostic: neutral trajectory type (Turn[]) and a plain async LLM callable at every boundary. No agent framework required.
• Pluggable store: in-memory InMemoryStore ships with the package; implement the Store interface for persistence (SQLite, Postgres+pgvector, Redis, anything).
• Pluggable embedder: MiniLMEmbedder (sentence-transformers MiniLM, 384-d, via @xenova/transformers) is the default and runs entirely in JS. Implement the Embedder interface to swap in OpenAI embeddings or anything else.
• Memory items are distilled strategies, not raw traces, per the paper: a title, a description, and an actionable lesson, derived from both successes and failures.

API surface

import {
  ReasoningBank,           // class — the high-level bank
  InMemoryStore,           // class — default store
  MiniLMEmbedder,          // class — default embedder
  mattsRun,                // fn — memory-aware test-time scaling
  judgeTrajectory,         // fn — success/failure judge
  distillMemories,         // fn — distill memories from a trajectory
  coerceTrajectory,        // fn — normalize input messages
  renderTrajectory,        // fn — render to plain text
  // prompt constants (override or inspect)
  INDUCTION_SUCCESS_PROMPT,
  INDUCTION_FAILURE_PROMPT,
  JUDGE_PROMPT,
  MATTS_CONTRAST_PROMPT,
  MERGE_PROMPT,
  // types
  type Turn,
  type Trajectory,
  type MemoryItem,
  type LLM,
  type Embedder,
  type Store,
} from "reasoning-bank";

ReasoningBank methods: retrieve, formatAsSystemBlock, ingestTrajectory, addManual, integrateCandidate, list, delete.

Paper

Ouyang et al., ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory, ICLR 2026. arXiv:2509.25140

This package is an independent implementation and is not affiliated with the paper's authors.

License

Apache-2.0. Copyright 2026 Rama Krishna Bachu.