agent.memo

Agent memory SDK based on a filesystem paradigm. Organizes memories, resources and skills as a hierarchical virtual filesystem with memo:// URIs — inspired by OpenViking.

Concepts

Why a filesystem paradigm?

AI agents need persistent memory that is inspectable, portable, and tool-friendly. A local filesystem satisfies all three: you can open any file in your editor, grep across memories, back them up with git, and pass paths directly to external tools (PDF parsers, image processors, embedding pipelines). The SDK adds structure on top — a URI scheme, tiered summaries, relations, and search — without hiding the files from you.

URI scheme

memo:// URIs map 1:1 to physical paths:

memo://user/preferences/coding.md  →  {workspace}/user/preferences/coding.md
memo://resources/api-docs.md       →  {workspace}/resources/api-docs.md

Directory meta files

Every directory can carry up to four meta files. Their content is always written by the caller (your agent or LLM) — the SDK only stores and retrieves them.

{workspace}/user/preferences/
├── .abstract.md      # L0: one-sentence summary, ~100 tokens
├── .overview.md      # L1: navigational overview, ~1-2k tokens
├── .relations.json   # typed links to other nodes
├── .meta.json        # arbitrary structured metadata (used internally by plugins)
├── coding.md         # L2: actual content file
└── communication.md

Tiered context loading

Loading everything into every LLM call is wasteful. The tier system lets you control context budget:

| Tier | Source | Typical use | |------|--------|-------------| | 0 | .abstract.md | Always-on system prompt — "what the agent knows exists" | | 1 | .overview.md | On-demand navigation — drill into a relevant area | | 2 | Full file content | Final retrieval — read the actual memory or document |

A typical agent loop: load tier-0 summaries at startup → search to find relevant nodes → load tier-2 content for those nodes only.

Installation

# npm
npm install @devxiyang/agent-memo

# pnpm
pnpm add @devxiyang/agent-memo

# yarn
yarn add @devxiyang/agent-memo

RipgrepSearchBackend (default) uses a bundled rg binary via @vscode/ripgrep — no manual installation needed.

Quick Start

The example below shows a coding assistant that remembers user preferences, caches reference documentation, and assembles focused context for each LLM call.

1. Initialize

import { FileMemo } from '@devxiyang/agent-memo'
import { AgentMemory } from '@devxiyang/agent-memo/memory'
import { AgentResource, UrlFetcher, LocalFetcher } from '@devxiyang/agent-memo/resource'

// FileMemo is the filesystem backend.
// The onWritten hook fires after every file write — use it to keep
// directory summaries up to date so tier-0 context stays accurate.
await using memo = new FileMemo({
  workspace: './agent-workspace',
  hooks: {
    onWritten: async (uri, parentDirs) => {
      for (const dir of parentDirs) {
        const nodes   = await memo.ls(dir)
        const summary = await llm.summarize(nodes)          // your LLM call
        await memo.writeMeta(dir, { abstract: summary })
      }
    },
    onError: (err, event) => console.error(`[memo:${event}]`, err),
  },
})

const memory   = new AgentMemory(memo)
const resource = new AgentResource(memo, [new UrlFetcher(), new LocalFetcher()])

2. Store user preferences

// First conversation — learn about the user
await memory.remember('memo://user/preferences/coding.md',
  'Prefers TypeScript with strict mode. Avoids classes, favors functional style. ' +
  'Uses pnpm and Vitest for testing.',
  { source: 'onboarding' },
)

await memory.remember('memo://user/preferences/communication.md',
  'Prefers concise answers. Dislikes excessive bullet points. ' +
  'Wants code examples for every non-trivial suggestion.',
  { source: 'onboarding' },
)

// Later — update as you learn more (createdAt is preserved automatically)
await memory.remember('memo://user/preferences/coding.md',
  'Prefers TypeScript with strict mode. Avoids classes, favors functional style. ' +
  'Uses pnpm and Vitest. Recently adopted Zod for runtime validation.',
  { source: 'conversation-18' },
)

3. Cache reference documentation

// Fetch and cache external docs with a TTL
await resource.fetch(
  'https://zod.dev/README.md',
  'memo://resources/zod-docs.md',
  { ttl: '7d' },
)

// Cache a local design doc
await resource.fetch(
  './docs/architecture.md',
  'memo://resources/architecture.md',
)

4. Assemble context for an LLM call

// --- System prompt (always cheap) ---
// Tier-0 loads .abstract.md for each directory — one-sentence summaries only.
// This tells the agent what it knows without burning context budget.
const systemContext = await memo.context(
  ['memo://user/preferences/', 'memo://resources/'],
  0,
)

// --- Targeted retrieval (on demand) ---
// The user asks about Zod. Search memories for anything relevant.
const hits = await memory.search('zod validation', {
  scope: ['memo://user/preferences/'],
  limit: 5,
})

// Load full content for matched memories + the cached Zod docs
const detailUris  = hits.map(h => h.uri)
const detailContext = await memo.context(detailUris, 2)

const zodEntry = await resource.read('memo://resources/zod-docs.md')

// Build the final prompt
const prompt = `
${systemContext}

--- Relevant memories ---
${detailContext}

--- Zod reference ---
${zodEntry?.content ?? '(not cached)'}

User: How should I validate an API response with Zod given my preferences?
`.trim()

const reply = await llm.chat(prompt)

5. Soft-delete and hard-delete

// Soft-delete: sets forgottenAt in frontmatter, file kept on disk
// recall() and search() skip it by default
await memory.forget('memo://user/preferences/communication.md')

// Hard-delete: removes the file entirely
await memory.purge('memo://user/preferences/communication.md')

// Expired or re-fetched resources
await resource.refresh('memo://resources/zod-docs.md')   // re-fetch from original URL
await resource.delete('memo://resources/architecture.md') // remove file + metadata

Binary Files

FileMemo implements the BinaryMemo interface for direct binary I/O:

// Write binary content
await memo.writeBinary('memo://media/avatar.jpg', imageBytes)

// Read binary content
const bytes = await memo.readBinary('memo://media/avatar.jpg')

// Or get the physical path to pass to external tools
const path = memo.toPath('memo://media/avatar.jpg')
await sharp(imageBuffer).toFile(path)
// onWritten hook fires automatically

For cached remote binary resources, use AgentResource.readBinary() — see the Plugins section.

API

new FileMemo(options: FileMemoOptions)

| Option | Type | Default | Description | |--------|------|---------|-------------| | workspace | string | required | Root directory | | hooks | MemoHooks | — | Lifecycle hooks | | search | SearchBackend | RipgrepSearchBackend | Search implementation | | mime | MimeDetector | MagicMimeDetector | MIME type detection | | watch | boolean | true | Auto-start filesystem watcher |

Memo interface

// Content
read(uri): Promise<string | null>
write(uri, content): Promise<void>
delete(uri): Promise<void>

// Directory
ls(uri): Promise<MemoNode[]>
mkdir(uri): Promise<void>
rmdir(uri, recursive?): Promise<void>

// Meta (caller manages content)
readMeta(uri): Promise<DirMeta>
writeMeta(uri, meta): Promise<void>

// Relations
link(uri, targets, reason?): Promise<void>
unlink(uri, target): Promise<void>
relations(uri): Promise<Relation[]>

// Search
search(query, options?): Promise<SearchResult[]>

// LLM context assembly
context(uris, tier: 0 | 1 | 2): Promise<string>

BinaryMemo interface

For binary file I/O. FileMemo implements this; plugins that handle binary resources depend on it.

readBinary(uri): Promise<Uint8Array | null>
writeBinary(uri, content: Uint8Array): Promise<void>

FilesystemAccess interface

Implemented by local filesystem backends. Consumed by plugins that need direct path access.

toPath(uri): string          // convert memo URI to absolute filesystem path

MemoLifecycle interface

Manages the lifecycle of the filesystem watcher. Consumed by application code.

watch(): void                // start watcher
unwatch(): Promise<void>     // stop watcher
[Symbol.asyncDispose]()      // used by `await using`

Hooks

interface MemoHooks {
  onWritten?(uri, parentDirs): void | Promise<void>
  onDeleted?(uri, parentDirs): void | Promise<void>
  onLinked?(uri, targets): void | Promise<void>
  onUnlinked?(uri, target): void | Promise<void>
  onError?(err, event): void
}

All hooks are fire-and-forget — the SDK does not await them. Errors are forwarded to onError.

Search Backends

// Default — bundled rg binary, fast
new RipgrepSearchBackend()

// Fallback — pure JS, no system dependencies
new SimpleSearchBackend()

MIME Detector

// Default — magic bytes detection via file-type
new MagicMimeDetector()

// Custom
class MyDetector implements MimeDetector {
  async detect(path: string): Promise<string | undefined> { ... }
}

Plugins

AgentMemory — semantic memory for agents

File format

Every memory is stored as a Markdown file with YAML frontmatter. The SDK manages three standard fields automatically; any extra fields you pass are persisted as custom frontmatter alongside the content.

---
createdAt: "2024-01-15T10:30:00.000Z"   # set on first write, preserved on updates
source: "conversation-42"               # optional: where this memory came from
forgottenAt: "2024-02-01T09:00:00.000Z" # set by forget(), absent until then
confidence: 0.9                         # any custom fields you add
---

Prefers TypeScript strict mode with explicit return types.

Directory layout

Memories are plain files — you choose the structure that makes sense for your agent:

{workspace}/
├── agent/
│   ├── .abstract.md              # L0 summary maintained by caller
│   ├── preferences/
│   │   ├── coding.md             # memo://agent/preferences/coding.md
│   │   └── communication.md
│   ├── facts/
│   │   ├── user-name.md
│   │   └── timezone.md
│   └── episodes/
│       ├── 2024-01-15.md
│       └── 2024-01-16.md

API

import { AgentMemory } from '@devxiyang/agent-memo/memory'

const memory = new AgentMemory(memo)

// Store a memory
// - createdAt is set automatically on first write, preserved on subsequent updates
// - extra fields in options are stored as custom frontmatter
await memory.remember('memo://agent/preferences/coding.md', 'Prefers TypeScript strict mode', {
  source:     'conversation-42',
  confidence: 0.9,
})

// Read a single memory
// Returns null if the file doesn't exist or forgottenAt is set
const entry = await memory.recall('memo://agent/preferences/coding.md')
// {
//   uri:         'memo://agent/preferences/coding.md',
//   content:     'Prefers TypeScript strict mode',
//   frontmatter: { createdAt, source, confidence }
// }

// List direct children of a directory
// Defaults to workspace root (memo://) if no URI provided
// For files: reads frontmatter.summary, does NOT read body
// For directories: reads .abstract.md if present
// Excludes forgotten memories by default
const items = await memory.list('memo://agent/')
// [
//   { uri, name, isDir: false, summary: 'TypeScript strict mode', forgottenAt?: string },
//   { uri, name, isDir: true,  abstract: 'Coding preferences' },
// ]

// With a summary stored at remember() time, list() is enough to navigate
// without reading every file's full content
await memory.remember('memo://agent/preferences/coding.md', fullContent, {
  summary: 'TypeScript strict mode, functional style, pnpm + Vitest',
})

// Full-text search across memories
const hits = await memory.search('TypeScript', {
  scope:            ['memo://agent/preferences/'],  // restrict to subtree (optional)
  limit:            10,
  excludeForgotten: true,                           // default: true
  source:           'conversation-42',              // filter by source field (optional)
})
// hits: MemoryEntry[]

// Soft-delete: sets forgottenAt in frontmatter, file stays on disk
// recall() and search() exclude it by default; can be recovered manually
await memory.forget('memo://agent/preferences/coding.md')

// Hard-delete: removes the file entirely, no recovery possible
await memory.purge('memo://agent/preferences/coding.md')

// Get the physical filesystem path (e.g. to pass to an indexing tool)
const path = memory.toPath('memo://agent/preferences/coding.md')

AgentResource — external resource caching

File layout

Each resource is stored as a file at the URI path. Metadata (source URL, fetch time, TTL, content type) is stored in the parent directory's .meta.json under the _resource key — no separate sidecar files.

{workspace}/
└── resources/
    ├── .meta.json                 # stores metadata for all files in this dir
    ├── api-docs.md                # memo://resources/api-docs.md  (text)
    ├── report.pdf                 # memo://resources/report.pdf   (binary)
    └── avatars/
        ├── .meta.json
        └── user.png               # memo://resources/avatars/user.png

The .meta.json for the resources/ directory looks like:

{
  "_resource": {
    "api-docs.md": {
      "source":      "https://example.com/api.md",
      "fetchedAt":   "2024-01-15T10:00:00.000Z",
      "expiresAt":   "2024-01-16T10:00:00.000Z",
      "contentType": "text/markdown"
    },
    "report.pdf": {
      "source":      "/local/reports/q4.pdf",
      "fetchedAt":   "2024-01-15T11:00:00.000Z",
      "contentType": "application/pdf"
    }
  }
}

Built-in fetchers

| Fetcher | Handles | Notes | |---------|---------|-------| | UrlFetcher | http://, https:// | Auto text/binary via Content-Type | | LocalFetcher | /abs, ./rel, ../rel, C:\win, C:/win | Copies file, MIME from extension |

UrlFetcher treats text/* and common application/* subtypes (json, yaml, xml, toml, graphql, javascript, typescript…) as text (UTF-8). Everything else is stored as binary.

API

import { AgentResource, UrlFetcher, LocalFetcher } from '@devxiyang/agent-memo/resource'

// Default: [new UrlFetcher()]. Pass custom list to extend or replace.
const resource = new AgentResource(memo, [new UrlFetcher(), new LocalFetcher()])

// Fetch and cache from any source the fetchers can handle
await resource.fetch('https://example.com/api.md',  'memo://resources/api-docs.md', { ttl: '1d' })
await resource.fetch('/local/reports/q4.pdf',       'memo://resources/report.pdf')
await resource.fetch('https://example.com/img.png', 'memo://resources/img.png',     { ttl: '7d' })

// Read cached text content
// Returns null if the resource was never fetched, or the TTL has expired
const entry = await resource.read('memo://resources/api-docs.md')
// {
//   uri:     'memo://resources/api-docs.md',
//   content: '# API Docs ...',
//   meta:    { source, fetchedAt, expiresAt?, contentType? }
// }

// Read cached binary content (images, PDFs, audio, etc.)
const bin = await resource.readBinary('memo://resources/report.pdf')
// {
//   uri:     'memo://resources/report.pdf',
//   content: Uint8Array,
//   meta:    { source, fetchedAt, expiresAt?, contentType? }
// }

// Get the physical filesystem path — for external tools (PDF parsers, image processors…)
const path = resource.toPath('memo://resources/report.pdf')

// Re-fetch from the originally stored source (respects new options if provided)
await resource.refresh('memo://resources/api-docs.md')
await resource.refresh('memo://resources/api-docs.md', { ttl: '7d' })  // extend TTL

// Check if a resource is missing or its TTL has expired
const stale = await resource.isStale('memo://resources/api-docs.md')

// Read metadata without fetching the file content
const meta = await resource.getMeta('memo://resources/api-docs.md')
// { source, fetchedAt, expiresAt?, contentType? }  or  null

// Remove the file and its metadata entry
await resource.delete('memo://resources/api-docs.md')

TTL format: 30s · 5m · 1h · 7d · 1y

Custom fetchers

Implement ResourceFetcher to support any source — S3, databases, internal APIs, etc. Fetchers are tried in order; the first one whose canHandle() returns true is used.

import type { ResourceFetcher, FetchMeta } from '@devxiyang/agent-memo/resource'
import { writeFile } from 'node:fs/promises'

class S3Fetcher implements ResourceFetcher {
  canHandle(source: string): boolean {
    return source.startsWith('s3://')
  }
  async fetch(source: string, destPath: string): Promise<FetchMeta> {
    const data = await downloadFromS3(source)        // your S3 logic
    await writeFile(destPath, data)
    return { contentType: 'application/octet-stream' }
  }
}

const resource = new AgentResource(memo, [new S3Fetcher(), new UrlFetcher()])
await resource.fetch('s3://my-bucket/model.bin', 'memo://models/model.bin')

Each plugin depends on interfaces, not the concrete FileMemo class:

• AgentMemory requires Memo & FilesystemAccess
• AgentResource requires Memo & BinaryMemo & FilesystemAccess

Filesystem Structure

{workspace}/
├── user/
│   └── memories/
│       ├── .abstract.md
│       ├── .overview.md
│       ├── .relations.json
│       ├── preferences/
│       │   ├── .abstract.md
│       │   └── coding.md
│       └── photos/
│           └── avatar.jpg
├── agent/
│   └── memories/
│       ├── cases/
│       └── patterns/
└── session/
    └── {session-id}/

License

MIT