ai-agent-local-memory
v0.4.0
Published
Neural associative memory plugin for OpenCode with cross-session recall, spreading activation, and magic-context-compatible context management
Maintainers
Readme
AIAgentLocalMemory
🇨🇳 中文版 README:README_CN.md
AI-Agent-Local-Memory is a project that brings together, in one on-device stack:
- Auditable, drift-resistant memory — every fact traceable to its source
- Smart context scheduling — the right memories at the right time
- Background pre-reasoning — the agent thinks ahead while idle
- Full reasoning-trace observability — nothing happens in a black box
- Local teacher–student self-distillation — the agent learns and improves on your machine
- Built-in commonsense & ethical alignment — a safety foundation, not an afterthought
The next-generation memory architecture for local-first, private, explainable, self-evolving agents.
Features
Agent context and entertprise memory management
- Unlimited context management — long history is transparently compressed into
<compartment>summaries by a background historian while the recent tail stays full-fidelity, so conversations don't hit a token wall. - Neural memory — an associative graph (nodes + weighted synapses, Hebbian learning, spreading activation, working memory) recalls by meaning and association instead of keyword match.
- Cross-session shared memory — one global memory graph is reused across every project and every session on the machine.
- Cross-machine memory sync — the memory graph replicates between machines through a Git-backed append-only operation log (conflict-free,
neural_syncpush / pull / status). - Cross-device session sync — the full raw OpenCode session (messages, tool calls, reasoning) can be replayed on a second machine via
neural_session_import, so you switch computers and keep writing where you left off. - Public and publishable Memory - You can publish your memory anywhere (like github) or get others' public memory and merge with your own agenet memory, also others can get your agent memory and merge with his.
- Multilingual & semantic search — SQLite FTS5 with
Intl.Segmenterfor CJK, plus optional embedding-based semantic recall through any OpenAI-compatible or Ollama endpoint. - Automatic session persistence — every conversation is mirrored to a readable Markdown transcript, and the underlying
opencode.dbis gzip-backed up daily to iCloud. - Live TUI sidebar — memory graph stats, sync state, context compression ratio, and LoRA training progress are visible directly inside the OpenCode sidebar.
- Coexistence with 3rd party context manager(magic-context) — auto-detects
@cortexkit/opencode-magic-contextand disables conflicting hooks while keeping the neural tools available.
Growing agent
- Growing local agent — an optional local LLM (Qwen3 14B via Ollama by default) that runs in three modes — Observer (silently learns from the server LLM), Student (auto-escalates when unsure), or Primary (fully autonomous) — with a LoRA fine-tune pipeline that trains on divergence-filtered Q&A pairs and rolls back on regression.
- Escalation on demand —
neural_ask_serverlets the local agent consult the server LLM with a structured[Reasoning] + [Answer]prompt and stores every response as a reusable experience node. - Human common sense Human common sense and moral is the real security ensuring. Agent will periodically ask for reading classic book like 《论语》 to get what is humanity and moral.
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ ☁️ Server LLM (Claude, GPT, etc.) │
│ ┌─────────────────────────────────────┐ │
│ │ Shared brain. Powerful but: │ │
│ │ • No personal memory │ │
│ │ • No individual growth │ │
│ │ • Treats everyone the same │ │
│ │ • Every request costs money │ │
│ └─────────────────────────────────────┘ │
│ ▲ consult when stuck │
│ │ │
│ 🧠 Your Local Agent (Local LLM + This Plugin) │
│ ┌─────────────────────────────────────┐ │
│ │ Your own brain. Grows with you: │ │
│ │ • Remembers YOUR projects │ │
│ │ • Learns YOUR patterns │ │
│ │ • Gets faster over time │ │
│ │ • Runs free, locally │ │
│ │ • Asks the "shared brain" only │ │
│ │ when truly stuck │ │
│ └─────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘Architecture
┌──────────────────────────────────────────────────────────────────┐
│ Adapter Layer (per-host) │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ OpenCode │ │ OpenClaw │ │ CLI/API │ │
│ │ Adapter │ │ Adapter │ │ (future) │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
├───────┼──────────────┼──────────────┼─────────────────────────────┤
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Core Engine │ │
│ │ • Neural Graph (nodes + synapses + Hebbian learning) │ │
│ │ • Spreading Activation + Working Memory │ │
│ │ • Context Manager (historian + compartments) │ │
│ │ • Experience Store (learned solutions) │ │
│ │ • Training Data Collector (learning pairs) │ │
│ └──────────────────────┬─────────────────────────────────────┘ │
├─────────────────────────┼─────────────────────────────────────────┤
│ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─┐ │
│ │ Local LLM Layer (optional, ollama / remote) │ │
│ • Observer: silently learns from server LLM │ │
│ │ • Student: answers with auto-escalation safety net │ │
│ • Primary: fully autonomous, escalates on demand │ │
│ │ • LoRA Fine-tuning Pipeline (auto-triggered) │ │
│ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─┬─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─┘ │
├─────────────────────────┼─────────────────────────────────────────┤
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Storage Layer │ │
│ │ • SQLite + FTS5 (Intl.Segmenter for CJK) │ │
│ │ • Operation Log (append-only, git-syncable) │ │
│ │ • Cross-device sync via Git │ │
│ └────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────┘Storage architecture — where every byte lives
Top = the big LLM the agent talks to. Bottom = the optional local small model. In between sits the context layer (reads the host's DB for real token usage) and our own neural graph, plus every place we keep verbatim original text, the sync/backup fan-out, and the training-data generator.
╔═══════════════════════════════════╗
║ BIG LLM (Opus / GPT / Kimi …) ║ ← the model the agent talks to
╚═════════════════┬═════════════════╝
│ prompt (compressed) ▲ / reply ▼
┌──────────────────────────────────────────┼──────────────────────────────────────────┐
│ CONTEXT LAYER (messages.transform) │ HOST DB (read-only, NOT ours) │
│ historian + compartments + tail render ───┼──reads──► opencode.db │
│ decides what to send the LLM │ (message.tokens.* → real usage, │
│ │ part table → historian input) │
└──────────────────────────────────────────┼──────────────────────────────────────────┘
│ store / recall
┌──────────────────────────────────────────▼──────────────────────────────────────────┐
│ NEURAL GRAPH graph.db (~/.local/share/ai-agent-local-memory/) │
│ ┌────────────────────────────┐ ┌────────────────────────────┐ │
│ │ nodes │ │ synapses (edges) │ │
│ │ episode ← verbatim* │ │ src,dst,type,weight │ │
│ │ fact/concept/assertion │ │ entity·lexical·semantic·… │ │
│ │ value/culture/experience │ └────────────────────────────┘ │
│ │ (*episode content, ≤2000 │ ┌────────────────────────────┐ │
│ │ chars, user/asst ONLY — │ │ compartments │ │
│ │ safePutNode rejects │ │ p1/p2/p3 summaries (NOT │ │
│ │ tool output) │ │ verbatim) start/end ord │ │
│ └────────────────────────────┘ └────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ nodes_fts (FTS5, unicode61) = Intl.Segmenter TOKENS, │ ← NOT the raw text; │
│ │ NOT verbatim. Joins back to nodes.content for原文. │ original lives in │
│ └────────────────────────────────────────────────────────┘ nodes.content ▲ │
└───────┬───────────────────────────────────────────────────────────────────┬──────────┘
│ every write also appended (embedding stripped) │ verbatim mirrors
▼ ▼
┌───────────────────────────────┐ ┌───────────────────────────────────────────────────┐
│ OPERATION LOG │ │ VERBATIM ORIGINAL-TEXT STORES │
│ sync/operations.jsonl │ │ transcripts/<sid>.md ← FULL text + TOOL OUTPUT │
│ append-only, JSONL │ │ + reasoning (what every ✂ stub tells the │
│ add/update/del × node/edge │ │ LLM to grep) │
└──────────────┬────────────────┘ │ episodes/<sid>.json ← full SessionData │
│ │ sync/opencode-sessions/<sid>.jsonl ← msg+parts │
│ git auto-push └───────────────────────────────────────────────────┘
▼
┌───────────────────────────────────────────────────────────────────────────────────────┐
│ EXTERNAL BACKUP / SYNC │
│ GitHub jackieju/myaimemorystore.git ← ONLY operations.jsonl + *.session.json meta │
│ (.gitignore excludes graph.db / *.db.gz / whole-session *.jsonl) │
│ backups/<timestamp>/ ← local full copy iCloud opencode-backup/opencode.db.gz │
│ neural_sync import <repoUrl> ← one-way merge another person's shared memory (no push) │
└───────────────────────────────────────────────────────────────────────────────────────┘
┌───────────────────────────────────────────────────────────────────────────────────────┐
│ TRAINING-DATA GENERATOR (feeds the local small model) │
│ training-pairs/pairs.jsonl (Q/A) · tool-calls.jsonl (tool decisions + optional CoT) │
│ │ export-training-data.ts (merge: experience nodes + pairs + tool-calls, dedup) │
│ ▼ │
│ lora-pipeline/training-data/{train,valid}.jsonl ──train──► adapters/<ts>/*.safetensors │
└───────────────────────────────────────────────────────────┬───────────────────────────┘
│ LoRA adapter
╔═══════════════════════════════════▼═════════════════╗
║ LOCAL SMALL LLM (optional, ollama qwen3:14b …) ║ ← we store NO
║ we store only the endpoint+model reference, ║ model weights,
║ not the model weights themselves ║ only config
╚══════════════════════════════════════════════════════╝Three things the diagram is careful about (and why):
- FTS holds tokens, not原文.
nodes_ftsstoresIntl.Segmenter-segmented tokens for CJK-aware search; the verbatim text lives innodes.content(and the transcript MD). Searching joins FTS hits back tonodes.content. - transcript MD ≠ episode node. The transcript
.mdis a byte-for-byte mirror including tool output + reasoning; the graph'sepisodenode stores only user/assistant text, truncated to 2000 chars —safePutNode's allow-list explicitly rejects tool output. So the graph never bloats with tool dumps, but nothing is lost: the full dump is in the transcript, which every compaction stub points the LLM to. - GitHub only gets the operation log. The
.gitignoreexcludesgraph.db,*.db.gz, and whole-session*.jsonl, so the git remote stays tiny (append-only ops replay to rebuild the graph). The DB itself goes to localbackups/and iCloud instead.
Knowledge routing — where does what you learn end up?
Every conversation you have through OpenCode with this plugin is split into two streams:
┌──────────────────────────────┐
│ You (user) ↔ Agent + LLM │
│ (a conversation) │
└──────────────┬───────────────┘
│
┌─────────────┴─────────────┐
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────────┐
│ Project-specific facts │ │ General reasoning & skills │
│ │ │ (optional) │
│ ───────────────────── │ │ ───────────────────────── │
│ • file paths │ │ • how to debug a │
│ • bug root causes │ │ race condition │
│ • config values │ │ • when to consult Oracle │
│ • naming conventions │ │ • how to structure a │
│ • decisions made │ │ good commit message │
│ • what breaks what │ │ • language-agnostic │
│ │ │ problem-solving patterns │
└────────────┬────────────┘ └───────────────┬─────────────┘
│ │
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────────┐
│ Global Memory Graph │ │ Local LLM (fine-tuned) │
│ (SQLite + FTS + │ │ (Qwen3 14B + LoRA) │
│ embeddings + git) │ │ │
│ │ │ Trained from Q&A pairs │
│ Recalled by │ │ captured in Observer / │
│ neural_recall, │ │ Student mode. Each │
│ spreading activation. │ │ auto-triggered LoRA run │
│ │ │ makes it a little more │
│ Shared across every │ │ capable, permanently. │
│ project on this │ │ │
│ machine and every │ │ │
│ machine you sync to. │ │ │
└─────────────────────────┘ └─────────────────────────────┘Every time you learn from an LLM, your local LLM is learning more. Project facts flow into the memory graph so future conversations recall them by association. General reasoning and problem-solving patterns flow into training pairs so, after enough LoRA cycles, the local model can handle familiar shapes of problems on its own — no cloud API required.
Enterprise Use Case: Shared Growing Intelligence
For large teams working on complex projects, the plugin creates a collective intelligence that grows across all team members:
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ 🏢 Large Project (hundreds of developers, years of history) │
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │Developer│ │Developer│ │Architect│ │ Support │ │ QA │ │
│ │ Alice │ │ Bob │ │ Carol │ │ David │ │ Eve │ │
│ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │
│ │ │ │ │ │ │
│ └───────────┴───────────┴───────────┴───────────┘ │
│ │ │
│ ┌───────────▼───────────┐ │
│ │ Shared Memory Graph │ │
│ │ (Git-synced SQLite) │ │
│ │ │ │
│ │ • Alice debugged the │ │
│ │ auth module → stored │ │
│ │ • Bob optimized the │ │
│ │ query → stored │ │
│ │ • Carol's architecture│ │
│ │ decisions → stored │ │
│ │ • David's customer │ │
│ │ patterns → stored │ │
│ └───────────┬───────────┘ │
│ │ │
│ ┌───────────▼───────────┐ │
│ │ Shared Local LLM │ │
│ │ (team ollama server) │ │
│ │ │ │
│ │ Learns from EVERYONE: │ │
│ │ • Debugging patterns │ │
│ │ • Code conventions │ │
│ │ • Domain knowledge │ │
│ │ • Customer issues │ │
│ └───────────────────────┘ │
│ │
│ Month 1: Everyone asks server LLM constantly (high cost) │
│ Month 3: Local LLM handles routine questions (cost ↓ 40%) │
│ Month 6: Local LLM knows the project deeply (cost ↓ 70%) │
│ Month 12: New hires get instant access to all accumulated knowledge │
│ │
│ Key benefits: │
│ • Eve asks "why does payment fail for JP users?" → local LLM recalls │
│ David's support experience + Alice's debugging notes + Bob's fix │
│ • New developer joins → immediately has access to team's entire │
│ problem-solving history without reading thousands of documents │
│ • No knowledge lost when someone leaves — their experience lives on │
│ │
└─────────────────────────────────────────────────────────────────────────────┘Packages
| Package | Description |
|---|---|
| @ai-agent-local-memory/core | Host-agnostic engine: graph, Hebbian learning, spreading activation, working memory, context renderer |
| @ai-agent-local-memory/storage-sqlite | SQLite + FTS5 storage (cross-runtime: Bun and Node.js) |
| @ai-agent-local-memory/adapter-opencode | OpenCode plugin adapter (full context management) |
| @ai-agent-local-memory/adapter-openclaw | OpenClaw plugin adapter (ContextEngine + memory slot) |
Ways to Use This Project
There are four ways to consume this project, from turnkey plugin to raw library. Pick the one that matches your host.
1. OpenCode plugin (turnkey, most features)
The richest integration. Full magic-context-style automatic context compression (historian + compartments), the growth layer (dreamer / observer-student-primary local LLM), the TUI sidebar, all neural_* tools, cross-machine sync, and session transcripts — all wired to OpenCode's plugin hooks.
- Install: see OpenCode Plugin → Install.
- Best for: anyone running OpenCode who wants everything with zero glue code.
2. OpenClaw plugin
The same neural memory core wired to OpenClaw's ContextEngine + memory slot. Shares the exact same memory graph as the OpenCode plugin by default (see Sharing Memory Between Hosts).
- Install: see OpenClaw Plugin → Install.
- Best for: OpenClaw users who want the memory core inside their existing setup.
3. nvp-server — JSON-RPC subprocess (any agent, any language)
A standalone compiled binary (packages/server/dist/nvp-server) that speaks JSON-RPC 2.0 over stdin/stdout (one JSON object per line, \n-framed). It depends only on core + storage-sqlite — zero OpenCode/OpenClaw dependency. Any agent that can spawn a subprocess and read/write pipes can use it (this is how the AI Voice Agent integrates it from Swift).
Build it:
cd packages/server
bun run build # produces dist/nvp-serverEvery request carries dbPath + projectId in params (the server keeps one engine instance per dbPath::projectId). Supported methods:
| Method | Purpose |
|---|---|
| ping | Health check → { ok, pid } |
| ingest | Feed one conversation turn: params.session = { id, messages: [{role, content, timestamp}] } (fire-and-forget) |
| recall | Associative recall: params = { query, maxResults?, maxHops?, threshold?, readOnly? } |
| remember | Store a node: params = { content, type?, importance?, metadata? } |
| renderContext | Return compressed history (fidelity-tiered): params = { sessionId, contextWindowTokens?, budgetRatio?, recentFullTextTurns?, seeds? } |
| getStats | Engine + working-memory stats |
| shutdown | Graceful stop |
Minimal example (feed a turn, then get compressed history back):
// → stdin
{"jsonrpc":"2.0","id":1,"method":"ingest","params":{"dbPath":"/path/graph.db","projectId":"default","session":{"id":"sess-1","messages":[{"role":"user","content":"hi","timestamp":1700000000000}]}}}
{"jsonrpc":"2.0","id":2,"method":"renderContext","params":{"dbPath":"/path/graph.db","projectId":"default","sessionId":"sess-1","contextWindowTokens":200000}}
// ← stdout
{"jsonrpc":"2.0","id":2,"result":{"messages":[{"role":"...","content":"..."}]}}- Best for: a custom agent (voice, CLI, web, another editor) in any language that wants memory + context rendering without writing an adapter.
- Note: the
nvp-serverpath currently uses the fidelity-tieredContextRenderer. The magic-context-style compartment compression lives in the OpenCode adapter today (see the two entry points below).
4. Use the source library directly (core + storage-sqlite)
Import the engine straight into your own TypeScript/JavaScript process — no subprocess, no RPC. This is what every adapter and nvp-server do under the hood.
import { NeuralContextEngine } from "@ai-agent-local-memory/core";
import { SqliteStorageProvider } from "@ai-agent-local-memory/storage-sqlite";
const storage = new SqliteStorageProvider({ storagePath: "/path/graph.db" });
const engine = new NeuralContextEngine();
await engine.init({ storage, projectId: "default", workingMemorySize: 1000, maxEdgesPerNode: 8 });
await engine.remember("The API base URL is localhost:6655", "fact");
const hits = await engine.recall("what is the api url", { maxResults: 5 });Other building blocks exported from core (compose your own pipeline): NeuralGraph, WorkingMemory, Historian, ContextRenderer, Compactor, Deduplicator, CrossSessionLinker, LightweightLinker, LLMExtractor, EmbeddingLinker, HierarchicalGraph, OperationLog, LoggedStorageProvider, EdgeWeightPredictor, plus the LLM/embedding providers under ./providers.
- Storage runs on both Bun (
bun:sqlite) and Node.js (node:sqlite) — see Cross-Runtime Compatibility. - Best for: embedding the memory engine inside your own service, or building a new host adapter.
At a glance
| Method | Runtime coupling | Context compression | Growth layer | Effort |
|---|---|---|---|---|
| OpenCode plugin | OpenCode only | compartment (magic-context style) | ✅ full | none |
| OpenClaw plugin | OpenClaw only | ContextEngine | partial | none |
| nvp-server (JSON-RPC) | any (subprocess) | fidelity-tiered (renderContext) | via remember/recall | spawn + pipe |
| Source library | any (in-process) | whatever you compose | whatever you compose | write your own glue |
Data Model
Nodes (Neurons): Memory units with types:
concept— Key entity extracted from conversationassertion— Composite claim from multiple conceptsdefinition— Definition-style descriptionfiller— Low-priority contextepisode— Full conversation referencemeta— Hub node (consolidated summary)fact— Durable note/fact that persists across sessionsexperience— Learned solution from server LLM consultation
Synapses (Edges): Weighted connections with types:
entity— Shared named entitytemporal— Co-occurrence in time windowlexical— Word overlap (Jaccard 0.2–0.55)semantic— Embedding similaritycausal— Causal relationshipcompositional— Composition (concepts → assertion)
How It Works
Hebbian Learning
Edges strengthen on co-activation: Δw = η × (1 - w) (asymptotic, never exceeds 1). Edges decay over time: w = w × exp(-λ × Δt). Weak, old, rarely-used edges get pruned.
Retrieval (Hybrid Scoring)
- Full-text search (FTS5, OR mode) → ranked by BM25
- Working memory boost for recently accessed nodes
- Spreading activation from top seeds → discover associated memories
- Hybrid score =
FTS_weight × fts_score + activation_weight × spread_score - Results sorted by descending relevance
Context Management (Historian + Compartments)
Long conversations are automatically compressed to stay within the context window:
- Recent messages (last 20) are kept at full fidelity
- Historian (background LLM) compresses older messages into compartments — summaries at 3 levels:
- p1 — Paragraph summary (~150 tokens): goals, decisions, files touched
- p2 — One sentence (~25 tokens): the most important thing that happened
- p3 — Title (~8 tokens): like a git commit subject
- Budget fitting: Compartments render at the highest fidelity that fits within 15% of context window
- Trigger: Every 6 turns or when context exceeds 80% budget, historian compresses the oldest uncompartmentalized window
Expanding compartments: When you see compressed history, say "expand that section" or "show me the original" — the LLM will call neural_expand(start=N, end=M) to retrieve the full original text.
Result: Infinite session support — context never overflows, old conversations are preserved as summaries, and original text is always retrievable on demand.
Viewing Original Conversation History
When conversations are compressed into compartments, you can always retrieve the original text:
Just ask naturally:
- "展开前面那段摘要"
- "让我看看之前讨论 X 的原文"
- "show me the full text of that compressed section"
- "expand the earlier conversation about Y"
The LLM sees compartments with ordinal markers (<compartment start="5" end="10">) and automatically calls neural_expand(start=5, end=10) to fetch the original messages from OpenCode's database.
Direct tool usage:
neural_expand(start=5, end=10)— expand a compartment by ordinal rangeneural_expand(tags="3-5")— expand by tag numberneural_session_read(sessionId="ses_xxx")— read messages from any session
Working Memory
LRU-frequency hybrid queue (default 1000 items). Score = frequency × exp(-0.01 × hours_since_access). Lowest-score items evicted when full.
OpenCode Plugin
Install
Option A: npm (recommended)
The plugin is published on npm as ai-agent-local-memory. OpenCode auto-installs npm plugins on startup — no manual npm install needed.
Note: The npm release may lag behind the latest development. If you want the most recent fixes and features, install from GitHub source (Option B below).
Add to your OpenCode config (~/.config/opencode/opencode.jsonc or project-level opencode.json):
{
"plugin": ["ai-agent-local-memory"]
}Restart OpenCode. On first launch, it will fetch the plugin from npm automatically.
To use a specific version (optional):
{
"plugin": ["[email protected]"]
}Option B: From source
git clone https://github.com/jackieju/AIAgentLocalMemory.git
cd AIAgentLocalMemory
bun install
bun build packages/adapter-opencode/src/index.ts --outdir dist --target bun --external @opencode-ai/plugin
mkdir -p ~/.config/opencode/plugins
cp dist/index.js ~/.config/opencode/plugins/ai-agent-local-memory.jsRestart OpenCode.
OpenCode Tools
| Tool | Description |
|---|---|
| neural_remember | Store a memory node (concept, assertion, definition, etc.) |
| neural_recall | Find memories by ASSOCIATION via graph traversal + spreading activation |
| neural_forget | Remove a memory node by ID |
| neural_note | Save durable facts/notes (session/project/global scope) |
| neural_reduce | Drop tagged content (suppress from rendering) |
| neural_pin | Pin content to always show at full fidelity |
| neural_expand | Expand compressed/elided content back to full text |
| neural_ask_server | Consult the server LLM for problems the local agent can't solve |
| neural_import_history | Import past OpenCode sessions into the neural graph |
| neural_session_import | Replay an OpenCode session exported from another machine (see Cross-Device Session Sync) |
| neural_backup | Backup the entire memory graph to a timestamped directory |
| neural_sync | Synchronize memory across machines via Git (init/push/pull/status) |
| neural_status | View engine stats and working memory |
OpenCode Configuration
Create neural-context.json in your project root or .opencode/ directory:
{
"injectSystemPrompt": true,
"contextWindowTokens": 128000,
"budgetRatio": 0.6,
"coexistWithOtherContextManager": false,
"syncRepo": "[email protected]:yourname/memory-sync.git",
"llm": {
"provider": "openai",
"model": "gpt-4o-mini"
},
"embedding": {
"provider": "openai",
"model": "text-embedding-3-small"
}
}| Option | Default | Description |
|---|---|---|
| injectSystemPrompt | true | Inject relevant memories into the system prompt each turn |
| contextWindowTokens | 128000 | Context window size in tokens for budget calculation |
| budgetRatio | 0.6 | Fraction of context window allocated to history |
| coexistWithOtherContextManager | auto-detected | Force coexistence mode on/off |
| syncRepo | — | Git remote URL for multi-machine sync (auto-initializes on startup) |
| llm.provider | — | LLM provider: "openai", "ollama", or "custom" |
| llm.baseUrl | https://api.openai.com/v1 | API endpoint (for custom providers) |
| llm.apiKey | $OPENAI_API_KEY | API key (or set via env var) |
| llm.model | gpt-4o-mini | Model name |
| embedding.provider | — | Embedding provider: "openai", "ollama", or "custom" |
| embedding.baseUrl | https://api.openai.com/v1 | API endpoint (for custom providers) |
| embedding.apiKey | $OPENAI_API_KEY | API key (or set via env var) |
| embedding.model | text-embedding-3-small | Model name |
LLM & Embedding Enhancement
When providers are configured, the engine automatically enhances memory quality:
| Module | Trigger | Effect |
|---|---|---|
| LLMExtractor | llm configured | High-quality concept/assertion extraction on ingest (vs regex) |
| EmbeddingLinker | embedding configured | Semantic edges via cosine similarity (vs lexical overlap only) |
| EdgeWeightPredictor | Always active | Multi-feature scoring improves edge weights |
| LightweightLinker | Always active | Baseline regex entity + lexical edges (no external service needed) |
Without providers configured, the system still works using LightweightLinker only.
Provider Examples
OpenAI:
{ "llm": { "provider": "openai", "apiKey": "sk-...", "model": "gpt-4o-mini" } }Ollama (local):
{ "llm": { "provider": "ollama", "model": "llama3.2" },
"embedding": { "provider": "ollama", "model": "nomic-embed-text" } }Custom OpenAI-compatible endpoint:
{ "llm": { "provider": "custom", "baseUrl": "http://localhost:8080/v1", "model": "my-model" } }Standalone Mode (e.g. replaces magic-context)
In standalone mode, this plugin fully manages the context window:
- Conversation history compression (activation-based fidelity rendering)
- Cross-session memory (neural graph with Hebbian learning)
- Session facts and notes
- Full context window budget management
Replace magic-context in opencode.json:
{
"plugin": ["ai-agent-local-memory"],
"compaction": { "auto": false, "prune": false }
}Coexistence Mode (e.g. alongside magic-context)
When magic-context is detected in your opencode.json, the plugin automatically enters coexistence mode:
messages.transformis disabled (magic-context handles context compression)- Memory content injection is disabled (avoids double injection)
- Tool usage guide is still injected (so the agent knows neural_* tools exist)
- All
neural_*tools remain fully functional for associative memory
In this mode, magic-context handles "context window management" while AIAgentLocalMemory provides "associative memory".
Per-project Control
| Scenario | How |
|---|---|
| Use alongside magic-context (default) | No config needed — auto-detected |
| AIAgentLocalMemory fully takes over | neural-context.json: {"coexistWithOtherContextManager": false} + remove magic-context from project opencode.json |
| Disable AIAgentLocalMemory for this project | Use a project-level opencode.json that doesn't load the plugin |
Growing Local Agent — Three Learning Modes
The plugin implements a growing local intelligence system: a local LLM progressively learns from a powerful server LLM (Claude/GPT) through observation, guided learning, or on-demand consultation. Over time, the local agent becomes increasingly self-sufficient.
Configuration
// ~/.config/opencode/neural-context.json
{
"localLlm": {
"provider": "ollama",
"endpoint": "http://localhost:11434", // or remote: "http://192.168.1.100:11434"
"model": "qwen3:8b",
"mode": "observer", // "observer" | "student" | "primary"
"confidence": {
"userThreshold": 0.5, // student mode: confidence below this triggers escalation
"autoEscalateAfter": 3 // student mode: auto-escalate after N user corrections
},
"training": {
"triggerCount": 100, // LoRA training triggers after this many training pairs
"cotStrategy": "none" // Reasoning capture: "none" (default, opt-in) | "thinking-tag" | "post-rewrite"
}
}
}Modes
| Mode | Main Model | Local LLM Role | Learning Method |
|---|---|---|---|
| observer | Server LLM (OpenCode config) | Silent observer | Stores every {question, reply} pair for learning |
| student | Local LLM (OpenCode provider = ollama) | Active with safety net | Auto-escalates via neural_ask_server when confidence is low |
| primary | Local LLM (OpenCode provider = ollama) | Fully autonomous | Only escalates when user explicitly says "问大模型" |
| (not configured) | Server LLM (OpenCode config) | N/A | Plugin works normally without local learning |
Observer Mode
The local LLM silently watches how the server LLM (Claude) handles every request:
User asks question → Claude answers → Plugin stores {question, answer} as training pair
→ After 100 pairs: triggers LoRA fine-tuning automatically- Training data stored in
~/.local/share/ai-agent-local-memory/training-pairs/pairs.jsonl - No impact on response quality — server LLM handles everything
- Ideal starting point: accumulate data before switching to student/primary mode
Reasoning capture (cotStrategy, opt-in)
Plain {Q, A} pairs teach the local model to imitate answer style but not reasoning. By default the plugin does NOT force any reasoning capture — every conversation stays exactly as the server LLM naturally responds and you never pay a token surcharge or wait for a rewrite. If you want reasoning in the training data too, you can opt in to chain-of-thought capture:
| Strategy | Behavior | User visibility | Cost |
|---|---|---|---|
| none (default) | No reasoning capture. Store {Q, A} as-is. | Nothing added to responses | Zero |
| thinking-tag | Server LLM wraps reasoning in <thinking>...</thinking> tags before the final answer. Both are stored as training output. | Hidden by most TUI themes, but does inflate response length | No extra API calls |
| post-rewrite | Original conversation runs normally. After session idle, a background sub-session asks the server LLM to rewrite the reply as [Reasoning] + [Answer] and stores the rewrite. | User sees the original reply as-is | One extra API call per Q&A pair (background) |
Configure via localLlm.training.cotStrategy. Default: none — reasoning capture is entirely optional. You decide which conversations get reasoning distilled and when.
Student Mode
The local LLM is the main responder (OpenCode provider = ollama), with automatic escalation:
User asks question → Local LLM assesses confidence
→ High confidence + has relevant experience: answers independently
→ Low confidence / unfamiliar topic: calls neural_ask_server → learns from response
→ User corrects 3+ times: auto-suggests escalation for subsequent questions- Confidence threshold configurable (
confidence.userThreshold, default 0.5) - Dissatisfaction detection: tracks "不对", "错了", "wrong", "重做" signals
- Every successful escalation stored as training pair → periodic LoRA fine-tuning
Primary Mode
The local LLM is fully autonomous — only escalates on explicit user request:
User asks question → Local LLM answers independently (always)
User says "问大模型" → calls neural_ask_server → learns from response- Maximum autonomy, minimum server LLM usage
- Ideal after significant LoRA training has been completed
LoRA Fine-Tuning Pipeline
Training happens automatically when enough data accumulates:
# Manual training (packages/lora-pipeline/)
./train.sh # MLX LoRA training (Qwen3 8B, rank 8, 200 iters)
./benchmark.sh # Compare base vs fine-tuned
./rollback.sh # Revert if degraded
# Or trigger from OpenCode:
# Use neural_export_training tool to export data manuallyAuto-training: Plugin monitors training pair count. When threshold is reached (100 for observer, 50 for student/primary), training is triggered in background with nice -n 19 (low CPU priority).
Sidebar status: TUI sidebar shows training status:
◆ LoRA Training
Last: 3h ago ✓ ← last training time + result
Runs: 5 Improved: 3/5 ← total runs + success countRemote Local LLM
The local LLM can run on another machine in your network:
{
"localLlm": {
"provider": "ollama",
"endpoint": "http://192.168.1.100:11434",
"model": "qwen3:32b"
}
}On the remote machine: OLLAMA_HOST=0.0.0.0 ollama serve
Sub-agent learning
Every time OpenCode goes idle, the plugin harvests training pairs not only from the main conversation but from every sub-session that was spawned during the turn — Oracle consultations, Explore/Librarian searches, Metis/Momus reviews, and any Sisyphus-Junior delegations. Each (user prompt, assistant reply) pair from a sub-session is written to pairs.jsonl with an instruction that flags it as a sub-agent style response ("reason step by step, cite evidence"). This way, the local LLM learns not just how the primary agent answers you, but how each specialist role thinks. Sub-session discovery is done via SELECT id FROM session WHERE parent_id = ? recursively (read-only) and capped at 50 sub-sessions per idle event for safety.
Replaying historical sessions
If you already have hundreds of past OpenCode sessions and want to mine them for training data right now (not wait for future conversations to accumulate), run the replay orchestrator:
# Replay every historical session (default: shortcircuit mode via forked opencode)
packages/lora-pipeline/replay-history.sh
# Replay 10 most recent sessions
packages/lora-pipeline/replay-history.sh --limit 10
# Replay only sessions from a given date onward
packages/lora-pipeline/replay-history.sh --since 2026-06-01
# Require at least 5 messages per session
packages/lora-pipeline/replay-history.sh --min-messages 5
# Fallback: use stock opencode with a read-only agent
packages/lora-pipeline/replay-history.sh --agent oracleThe script has two safety modes that both guarantee zero modifications to your local filesystem:
Mode 1 — Shortcircuit (default, highest reasoning fidelity):
- Uses a forked opencode binary at
~/.local/bin/opencode-forkfrom jackieju/opencode branch replay-shortcircuit, which adds an optionalshortcircuitfield to thetool.execute.beforeplugin hook. When a plugin sets it, opencode skips the real tool execution and returns that value as the tool_result. - Runs the complete Sisyphus agent — same system prompt, same permitted tools, same sub-agent dispatch (Oracle, Explore, Librarian, Metis, Momus, Sisyphus-Junior). Every reasoning step is identical to the original conversation.
- The plugin's
tool.execute.beforehook, activated by env varNEURAL_REPLAY_ORIG_SESSION_ID, queries the original session'sopencode.dbfor a completed tool call matching the current tool name + args, and returns the historical result via shortcircuit. Every Read, Grep, Edit, Write, Bash, WebFetch, etc. is served from history — the real tool is never executed. - A PR upstreaming the hook change to opencode: anomalyco/opencode#35613. Until it lands, use the fork.
Mode 2 — Read-only agent (fallback, if fork is missing):
- Uses stock opencode with
--agent oracle(or another opencode-defined read-only agent:plan,explore,librarian,metis,momus,multimodal-looker). - Opencode's runtime enforces the agent's tool whitelist — Edit / Write / Bash-write are hard-rejected, so nothing gets modified.
- Reasoning fidelity is lower because the agent's persona and available tools differ from the original Sisyphus run.
The orchestrator picks Mode 1 automatically when the fork binary exists; falls back to Mode 2 when you pass --agent; otherwise refuses to run.
Building the fork:
# One-time setup
git clone [email protected]:jackieju/opencode.git ~/Desktop/ju/projects/opencode
cd ~/Desktop/ju/projects/opencode
git checkout replay-shortcircuit
bun install
cd packages/opencode
bun run build --single --skip-embed-web-ui
# Symlink so replay-history.sh finds it
ln -sf ~/Desktop/ju/projects/opencode/packages/opencode/dist/opencode-darwin-arm64/bin/opencode ~/.local/bin/opencode-forkThe build takes ~2 minutes; the smoke test at the end prints Smoke test passed: 0.0.0-replay-shortcircuit-<timestamp>.
How replay works (both modes):
- Reads user message sequences out of your local
opencode.db(read-only, WAL-safe). - For each historical session, spawns a fresh headless opencode conversation in a scratch directory under
/tmp/replay-<sessionId>/. - Feeds the user messages through
opencode run --printone at a time (first with a fresh session, subsequent with--continue). - In Mode 1, every tool call is served from history via shortcircuit; in Mode 2, write tools are hard-rejected by the agent's whitelist.
- Every replayed assistant reply + every sub-agent call goes through the plugin's
session.idlecollector so it ends up inpairs.jsonl(main + sub-agent style). - When enough pairs accumulate, LoRA auto-train fires on its own.
Cost warning: replaying consumes real LLM API tokens exactly like a live conversation. Start with --limit 5 to gauge cost before running the full history. If you're on an Anthropic Pro/Max subscription this is effectively free (rate-limited only).
Replay is one-shot: it produces a one-time backfill of training data from history. From that point on, session.idle sub-agent harvesting keeps the training set growing incrementally.
Known limitation: opencode run --print is a headless one-shot invocation that may not always fire the session.idle event our collector relies on. End-to-end training-pair harvest during replay is under active integration — the safety mechanism (no local modifications) has been verified, but pairs.jsonl growth during a headless replay may lag until we wire an alternate trigger. Live interactive sessions continue to collect pairs normally.
Progression Path
1. Start with observer mode (accumulate 100+ training pairs)
2. Run LoRA fine-tuning on accumulated data
3. Switch to student mode (local LLM with safety net)
4. As local model improves, reduce escalation frequency
5. Switch to primary mode (fully autonomous local agent)OpenClaw Plugin
Install
Option A: From source (recommended for development)
git clone https://github.com/jackieju/AIAgentLocalMemory.git
cd AIAgentLocalMemory
bun install
rm -rf packages/adapter-openclaw/node_modules
bun build packages/adapter-openclaw/src/index.ts --outdir packages/adapter-openclaw/dist --target node --external "openclaw" --external "bun:sqlite" --external "node:sqlite"
openclaw plugins install --force packages/adapter-openclaw
openclaw gateway restartTo update after code changes:
bun build packages/adapter-openclaw/src/index.ts --outdir packages/adapter-openclaw/dist --target node --external "openclaw" --external "bun:sqlite" --external "node:sqlite"
cp packages/adapter-openclaw/dist/index.js ~/.openclaw/extensions/neural-context/dist/index.js
openclaw gateway restartOption B: npm (when published)
openclaw plugins install @ai-agent-local-memory/adapter-openclaw
openclaw gateway restartOpenClaw Configuration
Add to ~/.openclaw/openclaw.json:
{
"plugins": {
"entries": {
"neural-context": {
"enabled": true,
"hooks": {
"allowPromptInjection": true,
"allowConversationAccess": true
},
"config": {
"autoRecall": true,
"autoCapture": true,
"maxRecallResults": 10
}
}
},
"slots": {
"memory": "neural-context",
"contextEngine": "neural-context"
}
}
}| Option | Default | Description |
|---|---|---|
| storageDir | ~/.local/share/ai-agent-local-memory-openclaw | Custom storage directory |
| autoRecall | true | Inject relevant memories before every AI turn via spreading activation |
| autoCapture | true | Store conversations and build associative edges after every turn |
| maxRecallResults | 10 | Maximum memories injected into context per turn |
| debug | false | Enable verbose debug logs |
OpenClaw Context Engine
The plugin registers as a full ContextEngine (ownsCompaction: true), providing:
- assemble() — Activation-based fidelity rendering (f0-f4) within token budget
- compact() — Delegates to OpenClaw's built-in compaction as fallback
- ingest() / afterTurn() — Stores messages as episode nodes with automatic edge creation
- bootstrap() — Initializes storage on session start
This means conversation history is managed intelligently — old messages are compressed based on relevance (not just age), and re-activate to full fidelity when their topic becomes relevant again.
OpenClaw Tools
| Tool | Description |
|---|---|
| neural_recall | Find memories by ASSOCIATION via graph traversal + spreading activation |
| neural_remember | Store information with automatic associative linking |
| neural_forget | Remove a memory node by ID |
| neural_note | Save durable facts/notes that persist across sessions |
| neural_status | View engine stats and working memory |
Storage
Default Paths
| Adapter | Default Storage Path |
|---|---|
| OpenCode | ~/.local/share/ai-agent-local-memory/ |
| OpenClaw | ~/.local/share/ai-agent-local-memory-openclaw/ |
Storage contents:
├── graph.db ← all nodes, edges, FTS index (single SQLite file)
├── graph.db-wal ← WAL journal (may not exist when idle)
├── graph.db-shm ← shared memory (may not exist when idle)
├── episodes/ ← raw session JSON files (original conversation text)
├── transcripts/ ← auto-synced chat transcripts (one file per session)
└── backups/ ← created by neural_backup toolCustom Storage Path
OpenCode — neural-context.json or env var:
{ "storageDir": "/path/to/custom/storage" }export AI_AGENT_LOCAL_MEMORY_DIR=/path/to/custom/storageOpenClaw — plugin config in ~/.openclaw/openclaw.json:
{ "plugins": { "entries": { "neural-context": { "config": { "storageDir": "/path/to/custom/storage" } } } } }Sharing Memory Between Hosts
Point both adapters to the same directory to share a single memory graph:
// OpenCode neural-context.json
{ "storageDir": "~/.local/share/ai-agent-shared-memory" }
// OpenClaw plugin config
{ "storageDir": "~/.local/share/ai-agent-shared-memory" }SQLite WAL mode handles concurrent reads safely. Concurrent writes are rare and retried via busy timeout.
Distributed Sync (Multi-Machine)
Synchronize memory across multiple machines using Git. Uses an append-only operation log — no conflicts possible.
How it works
Machine A writes → appends to operations.jsonl → git push
Machine B: git pull → replay new operations → local graph updatedEach machine appends its own operations. Git merges are always clean (append-only, no overlapping lines). UUID node IDs prevent collisions across machines.
Setup
Machine 1 (first time):
neural_sync(action="init", repoUrl="[email protected]:yourname/memory-sync.git")Machine 2 (join existing):
neural_sync(action="init", repoUrl="[email protected]:yourname/memory-sync.git")
neural_sync(action="pull")Daily Use
neural_sync(action="push") → commit + push local operations
neural_sync(action="pull") → pull + replay remote operations
neural_sync(action="status") → check sync stateWrites During Sync
If new memories are created while syncing, they are safely appended to the operation log and included in the next push. No data loss is possible.
Architecture
graph.db (local runtime database — fast reads/writes)
↑ replay
operations.jsonl (append-only log — synced via Git)The operation log is the source of truth for sync. graph.db is a materialized view that can be rebuilt from the log at any time.
Backup
Session Transcripts
Every time OpenCode finishes responding (session enters idle state), the full chat history is automatically written to:
~/.local/share/ai-agent-local-memory/transcripts/<sessionId>.mdThe file stays in sync with the session — each idle event checks for new content and updates the file. Format:
[user] How do I fix the auth bug?
---
[assistant] Looking at the auth module...
---
[tool] { result of tool call }
---This gives you a persistent, readable copy of every conversation that survives even if the session is deleted from OpenCode.
Cross-Device Session Sync
Move to a new machine and continue the same OpenCode session — including the full message history, tool calls, and reasoning — as if you never left.
How it works (design):
- Append-only export: On every
session.idleevent, the plugin reads OpenCode's localopencode.db(SQLite, WAL mode, read-only handle, no write contention with OpenCode's own writes) and appends any new messages/parts to~/.local/share/ai-agent-local-memory/sync/opencode-sessions/<sessionId>.jsonl. - Per-session cursor:
.exporter-state.jsonremembers the last exportedmessage_idper session, so subsequent exports are incremental — one idle event only writes the delta. - First-time cap: A brand-new session on this machine exports at most 500 messages per idle event to avoid a large blocking write; the rest is picked up on later idle events.
- Git-backed sync: The
opencode-sessions/directory lives inside the same Distributed Sync repo (syncRepo). The existing background sync timer commits and pushes them along with the neural graph — no extra network round-trip. - Structured replay: Each JSONL line is
{"msg": <message row>, "parts": [<part row>, ...]}— a lossless snapshot of the OpenCode schema, not a text transcript. Replay recreates the exact same conversation OpenCode would render. - Session metadata: A one-time
<sessionId>.session.jsoncaptures title, directory, model, and token stats so the imported session appears correctly in OpenCode's session list.
Non-blocking by design:
- Never blocks the event loop: only async I/O, never
execSync. - Never blocks
messages.transform: exports run on thesession.idleevent, after the assistant has already replied. - Never writes to
opencode.db: read-only handle, so SQLite lock contention with OpenCode is impossible. - Piggy-backs on the sync timer: no separate git process, no extra network chatter.
Usage — export (automatic, no action required):
Whenever OpenCode goes idle, new messages of the active session are appended to the JSONL and, on the next sync tick, pushed to the sync repo.
Usage — import on a new machine:
Configure
syncRepoon the new machine so its neural memory pulls from the same git repo (see Setup).Wait for the sync timer to pull (or run
neural_syncaction=pull) — this brings theopencode-sessions/directory to the new machine.From inside OpenCode, run the tool:
neural_session_import(sessionId="ses_...")Optional:
overwrite=trueto re-insert messages even if some already exist locally (default is safe idempotent replay — duplicates are skipped viaINSERT OR IGNORE).Restart OpenCode so it re-reads
opencode.db— the imported session now appears in the session list and can be reopened withopencode --session <sessionId>.
Storage layout in the sync repo:
<syncRepo>/opencode-sessions/
├── ses_abc123.jsonl append-only messages + parts
├── ses_abc123.session.json one-time session metadata
├── ses_xyz789.jsonl
├── ses_xyz789.session.json
└── .exporter-state.json per-session cursor (last exported message_id)Feature description:
You keep OpenCode's native single-machine session UX. Behind the scenes, every idle moment writes the incremental diff to a git-synced JSONL, so any second machine (macOS / Linux / Windows) that pulls the same repo can call one tool and pick up exactly where you left off — same messages, same tool history, same reasoning traces.
Via Tool (in OpenCode)
neural_backup() → ~/.local/share/ai-agent-local-memory/backups/<timestamp>/
neural_backup(destination="/path/to/dir") → custom pathManual
cp -r ~/.local/share/ai-agent-local-memory/ ~/backup/ai-agent-local-memory-$(date +%Y%m%d)/Development
bun install
bun build packages/adapter-opencode/src/index.ts --outdir packages/adapter-opencode/dist --target bun --external @opencode-ai/plugin
bun build packages/adapter-openclaw/src/index.ts --outdir packages/adapter-openclaw/dist --target node --external "openclaw" --external "bun:sqlite" --external "node:sqlite"Cross-Runtime Compatibility
The storage layer uses a sqlite-shim that automatically selects:
bun:sqlitewhen running under Bun (OpenCode)node:sqlite(DatabaseSync) when running under Node.js (OpenClaw)
No external native dependencies required on either runtime.
Development Session
Original design session (OpenCode):
opencode --session ses_166d0e7b9ffeBpCjAtqVkPPkP4Documentation
Further reading in docs/:
- Configuration Reference (中文) — every configurable option, with defaults and a full example.
- Context Compression Pipeline (中文) — the full flow from a user keypress to the assembled LLM payload.
- Compression Strategies (A/B/C/D) (中文) — the four selectable strategies, their trade-offs, and when to use each.
- Commonsense Foundation Spec — how the agent grows commonsense and ethics as its safety layer.
- Architecture & Safety Whitepaper — system architecture and the safety model.
- Working with a Local LLM — a walkthrough of the Observer / Student / Primary local-LLM modes.
License
Copyright (C) 2026 Jackie Ju
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.
