mnema

(Greek μνῆμα — memory, memorial. Same root as Mnemosyne.)

File-first personal memory layer for AI agents. Markdown is the source of truth; SQLite is a disposable index. Conflicting memories are recorded, never auto-deleted — resolution is a human decision, made when it is cheap to ask.

Born from a source-level audit of existing memory systems (memmy-agent, Honcho, mem0). They converge on hybrid retrieval and LLM extraction — and diverge on exactly the things mnema bets on: human-readable storage, mandatory provenance, and honest conflict handling.

Why another memory system

| Problem in existing systems | mnema's answer | |---|---| | Memory locked in a DB you can't grep, diff, or edit | Markdown files are canonical; the SQLite index rebuilds from them | | Contradictions silently accumulate, or an LLM silently deletes the "old" fact | Conflicts become records with provenance; a human resolves; losers are downweighted, never deleted | | Memories can't be traced back to their source | Every derived memory carries the session ref, message ids, and a redacted excerpt inline — auditable even after transcripts are purged | | Everything gets stored, quality control deferred to retrieval | Extraction is an explicit, human-confirmed distill step with a write-time filter |

Status: working, pre-release

Everything under Implemented is covered by the test suite (50 tests, offline and deterministic) and has been exercised live against real Claude Code sessions and the real Anthropic API.

Quick start

npm install -g @raytien/mnema   # or: pnpm install && pnpm build (from source)
mnema --help

# manual memory
node dist/cli.js add --body "Always use pnpm, never npm" --stable

# hybrid search (first run downloads a ~118MB local embedding model;
# set MNEMA_NO_EMBED=1 for keyword-only, zero download)
node dist/cli.js search "package manager"

# extract memories from a Claude Code session (needs ANTHROPIC_API_KEY)
node dist/cli.js distill ~/.claude/projects/<proj>/<session>.jsonl --output run.json
# review run.json, then:
node dist/cli.js distill --apply run.json

MCP (read-only search from any Claude Code session):

claude mcp add mnema -- mnema-mcp

Env: MNEMA_ROOT (default ~/.mnema), MNEMA_NO_EMBED=1, MNEMA_ENABLE_WRITE=1 (MCP write tools), MNEMA_MODEL, ANTHROPIC_API_KEY.

Recommended: cd ~/.mnema && git init — your memory history is just files.

Implemented

Storage — files first, crash-safe

• Canonical markdown memories with Zod-validated frontmatter (ULID ids, versioned schema, discriminated source union)
• Write protocol: intent journal → cross-process lock → manifest generation → atomic file write (fsync + rename) → DB transaction. Fault-injection tests cover every crash point; recovery replays the journal without re-calling any LLM
• Repair-before-read across processes (durable dirty marker, not in-memory state); in-place index rebuild that never unlinks an open DB
• Manual edits detected by content hash: revision bumps, source wraps as revised, embeddings recompute — automatically, on the next index
• op_key/op_hash idempotency: same key replays, same key with a different payload errors (never silently dropped)

Retrieval — hybrid, multilingual

• FTS5 (contentless-delete, remove_diacritics 2) + local vector search (sqlite-vec, pinned multilingual MiniLM, q8) fused with RRF
• Shared tokenizer for index and query: Intl.Segmenter + Han bigrams — Chinese two-character terms actually hit (raw unicode61 scores zero); English/Spanish/Portuguese/French work as-is; cafe finds café
• Query hardening: user input never reaches FTS MATCH raw (C++, alpha -beta, emoji-only queries are all safe); input caps on every untrusted surface
• Time decay after fusion (stable memories exempt); superseded memories downweighted, derived from the resolution graph
• Cross-lingual retrieval via multilingual embeddings (verified live: English queries matching Chinese memories)

Distill — explicit, audited capture

• Claude Code JSONL parsed defensively (no public schema; bad lines counted, never a crash)
• Versioned redaction runs before anything reaches the LLM; secrets never survive into stored excerpts
• One extraction call per preview; write-time filter (preferences, decisions, constraints only — empty sessions honestly yield zero)
• run.json is immutable (tamper-detected by hash): accept/reject only; apply is LLM-free and idempotent
• Fabricated citations are dropped — provenance must be real

Conflicts — record, never auto-delete

• Batched LLM judging of semantically-near pairs; verdicts stored as deterministic relation files with the input hashes they were judged on
• Relations go stale when a member is edited, orphaned when deleted; stale verdicts stop affecting ranking
• resolve keep:<id> | keep_both re-verifies hashes under lock and rejects supersession cycles (A>B>C>A)
• Judge failures are recorded (error) and retryable — a network blip never becomes a permanently missed conflict
• Unresolved conflicts surface alongside search results

Interfaces

• CLI: add / search / distill / resolve / check-conflicts / index / eval
• MCP over stdio: read-only search by default; add and two-phase resolve (preview token required to commit) only behind an explicit flag
• Eval harness with a draft gold set; deterministic FTS-only baseline pinned in CI (recall@3 0.70, gate at 0.62)

Security posture

• 0700/0600 permissions, atomic temp-file writes, stdio-only MCP
• Honest residual risk: a poisoned conversation distilled into memory is persistent prompt injection; the mitigation is the human confirm step in distill, not a technical control

Roadmap

Near-term (blocked on a confirmed gold set):

• [ ] Calibrate RRF k, decay half-life, and the conflict-candidate distance threshold (current values are literature defaults)
• [ ] Held-out test set and scheduled (non-CI) LLM quality evals: distill precision/recall, redaction precision, contradiction F1 with negative pairs

Planned:

• [ ] mnema review — batch conflict triage in the terminal
• [ ] Distill sources beyond Claude Code (Cursor, Codex session formats)
• [ ] Freshness re-verification (source_status) against still-existing transcripts
• [ ] Custom FTS tokenizer preserving symbol terms (C++ vs C — currently a documented limitation; recall via the vector path only)
• [ ] npm publish + prebuilt binary matrix (macOS arm64, Linux x64)

Explicitly out of scope (v1 promises, not omissions):

• No agent runtime, no desktop app, no background daemon — one CLI, one MCP server, LLM calls only in explicit steps
• No auto-deletion of memories, ever
• No multi-user / workspace / auth — a personal, local tool
• Japanese/Korean text: detected and warned, not usefully indexed

Design history

The full design history (comparison matrix, reviewed plan, implementation specs, milestone tracking) is maintained privately.