rlm-pi

RLM (Recursive Language Model) extension for PI (pi-mono) — a persistent Python kernel, a context lake, and background subagents that keep large data out of the LLM prompt.

RLM is a research paradigm introduced by Alex Zhang (October 2025) and formalized in the paper Recursive Language Models (arXiv:2512.24601). This extension implements that paradigm for PI.

Companion project: rlm-opencode (the same RLM for OpenCode). Both share the same kernel runtime and the same context-lake format, so state is portable between editors.

Why RLM?

LLM agents degrade as context grows: cost rises linearly, performance drops ("context rot"), and every turn re-sends the same data. RLM treats context as variables instead of stuffing everything into the prompt:

Traditional:  Context (huge) → prompt → LLM → 💀 context rot, cost explosion
RLM:          Context → kernel / lake (external) → LLM calls tools to get only what it needs → ✅

The model keeps working state in a persistent Python kernel and large reference data in a context lake, then queries both with small tool calls. The big data never enters the prompt.

When does RLM help most?

| Situation | Without RLM | With RLM | |---|---|---| | Logs, build output, CSV dumps (100KB–10MB) | Prompt overflow or huge token bills | Stored in kernel/lake; model queries snippets | | Multi-step data pipelines (transform → analyze → report) | Every step re-reads and re-sends data | Variables survive in the kernel between steps | | Long agent sessions (compaction) | State lost or re-summarized | Snapshot + variable summary at compaction | | Local models with small context (1–8K) | Cannot process large data at all | Model writes code; kernel does the heavy lifting | | Parallelizable review/analysis tasks | Serial, blocking | Background subagents with admission handles |

Impact study: local models vs frontier models (measured 2026-09-06)

Same task in both modes — find NEEDLE-7A3F in a 5,000-line log file (~280 KB ≈ 130,000 tokens):

• Mode A (prompt): the data is injected into the LLM prompt.
• Mode B (RLM): the model only writes a small Python snippet; the kernel executes it against the file.

| Model | Type | Mode A (data in prompt) | Mode B (RLM) | |---|---|---|---| | LFM2.5-1.2B (local) | small | ❌ Prompt too long: 130,034 tok > 128K ctx | ✅ 75 tok · 0.6s | | LFM2.5-2.6B (local) | small | ❌ Prompt too long: 130,033 tok | ✅ 73 tok · 4.3s | | LFM2.5-8B-A1B (local) | mid | ❌ Prompt too long: 130,032 tok | ✅ 72 tok · 3.7s | | Qwen3.6-35B-A3B (local) | large local | ❌ OOM: memory guard aborted prefill (27.9 GB) | ✅ (needs free RAM) | | deepseek-v4-flash (cloud) | frontier | ✅ 125,106 tok · 7.4s | ✅ 387–1,374 tok |

What this proves:

1. Small local models cannot process large data in the prompt — not even a 128K-context server: the file exceeds the window. With RLM they succeed with 72–75 tokens in under 5 seconds.
2. Large local models cannot either — the prefill of 130K tokens needs ~21–28 GB of KV cache + attention, which trips the memory guard.
3. Frontier models can (125,106 tokens, 7.4s) — but they pay 125,106 tokens per request. With RLM the same task costs 387–1,374 tokens: a 90–1,700× reduction.
4. RLM is therefore not an optional optimization for small local models — it is what makes them capable of working with real data. The model becomes an orchestrator of code, not a reader of documents.

Tools

| Tool | What it does | |---|---| | ipython | Execute Python in a persistent kernel. Variables, imports, functions and results survive across calls. %%bash for shell, %cd for persistent cwd, top-level await, timeouts, output caps. | | rlm_store / rlm_get / rlm_search / rlm_find / rlm_stats / rlm_forget | Context lake — large data stored here never enters the LLM prompt; the model retrieves only what it needs. | | rlm | Spawn a background subagent (pi --mode json "<prompt>") and get an admission handle immediately. | | rlm_result / rlm_list | Child results and listing. |

Install

Requirements: PI ≥ 0.70, Python 3.9+ (3.11+ recommended, stdlib only).

Via npm (published as rlm-pi):

npm install -g rlm-pi
rlm-pi-install   # copies extension + kernel to ~/.pi/agent/
# restart pi

Via git:

git clone https://github.com/nicolasramos/rlm-pi.git
cd rlm-pi
# 1. Copy the extension
mkdir -p ~/.pi/agent/extensions
cp rlm.ts ~/.pi/agent/extensions/rlm.ts

# 2. Copy the kernel
mkdir -p ~/.pi/agent/rlm-kernel
cp kernel.py ~/.pi/agent/rlm-kernel/kernel.py

# 3. Reload pi (/reload) or restart

Or install as a package:

pi install rlm-pi

Optional env vars: RLM_KERNEL (kernel path), RLM_KERNEL_PYTHON (interpreter).

Usage — complete documentation

1. Persistent kernel (ipython)

The kernel is a durable CPython REPL. Everything you define survives across calls — variables, imports, functions, even the working directory.

Basic state persistence:

> Store the first 5 primes in a variable, then double the last one.

⚙ ipython {"code": "primes = [2, 3, 5, 7, 11]"}
⚙ ipython {"code": "primes[-1] * 2"}
→ 22

Shell access with %%bash:

⚙ ipython {"code": "%%bash\ngit log --oneline -5"}
→ 9d29dede2 feat: add HermesApp iOS scaffold
  772be82c7 auto-sync 2026-07-22 18:21
  ...

Persistent working directory with %cd:

⚙ ipython {"code": "%cd /tmp/project"}
→ /tmp/project
⚙ ipython {"code": "import os; os.getcwd()"}
→ '/tmp/project'

Top-level await:

⚙ ipython {"code": "import asyncio; await asyncio.sleep(0.1); 'done'"}
→ 'done'

Output caps: stdout is capped at 100KB and expression reprs at 4K chars, so the model receives concise results and queries specific values with small cells instead of dumping data into the conversation.

2. Context lake (rlm_store, rlm_get, rlm_search, rlm_find, rlm_stats, rlm_forget)

Large data lives in a per-project JSONL store. The model stores it once and queries snippets on demand — the data never enters the prompt.

> Store the full build log in the lake, then find the failing test.

⚙ ipython {"code": "logs = open('build.log').read()"}          # 2MB in the kernel
⚙ rlm_store {"key": "build-log", "content": "logs"}            # 2MB in the lake, NOT in the prompt
⚙ rlm_search {"pattern": "FAILED|error"}                       # snippets only
→ 1. build-log (2,048,000 chars)
   …FAILED test_api_orders — AssertionError: expected 200…

⚙ rlm_get {"key": "build-log"}                                 # full entry (capped 50KB)
⚙ rlm_find {"query": "AssertionError"}                         # text search
⚙ rlm_stats {}
→ {"entries": 12, "total_chars": 2_500_000, "file": "..."}
⚙ rlm_forget {"key": "build-log"}
→ {"deleted": true}

From inside the kernel — the rlm_lake module writes a kernel variable straight into the lake without any prompt round-trip:

⚙ ipython {"code": "rlm_lake.store('biglogs', logs)"}          # 200KB → lake, 0 tokens spent

3. Background subagents (rlm, rlm_list, rlm_result)

Spawn a child PI process that works in the background while you keep going. You get an admission handle immediately (RLM semantics); results arrive later.

> Review the auth flow in the background while I keep working.

⚙ rlm {"prompt": "Review the authentication flow for security issues", "name": "auth-reviewer"}
→ {"rlm_child_id": "...", "name": "auth-reviewer", "status": "running"}

⚙ rlm_list {}
→ [{"name": "auth-reviewer", "status": "running", "created": "..."}]

⚙ rlm_result {"child": "auth-reviewer"}
→ {"status": "completed", "output_tail": "..."}

rlm spawns pi --mode json "<prompt>" as a subprocess and streams its JSON events to a file; rlm_result reads the tail.

How it works

• kernel.py is a self-contained CPython REPL speaking newline-delimited JSON over stdio (same runtime as rlm-opencode). One kernel per project directory, spawned lazily on first ipython use.
• The context lake is a per-project JSONL store at ~/.pi/agent/rlm-state/lake/ — identical format to rlm-opencode's lake, so data written by one editor is readable by the other.
• rlm spawns pi --mode json "<prompt>" as a subprocess and streams its JSON events to a file; rlm_result reads the tail.

Verified (real pi CLI, v0.73.1, model opencode-mix-deepseek-v4-flash via LiteLLM)

| Test | Result | |---|---| | ipython state persistence (two separate calls) | secret_number = 12345 → secret_number * 2 → 24690 ✅ | | Context lake (50K log lines, ~2.3MB) | stored via rlm_store, needle REQ-43210 found via rlm_search ✅ | | Tool discovery | model lists all RLM tools (ipython, rlm_store/get/search/find/stats/forget, rlm, rlm_result, rlm_list) ✅ |

Security

The kernel executes model-generated Python with your OS permissions. It is a durable control environment, not a security sandbox. Use an external sandbox for untrusted repositories.

FAQ

Does RLM replace the model's context window? No — it complements it. The context window is reserved for reasoning and instructions; data lives in the kernel/lake.

Does it work with any model? Yes. RLM is model-agnostic: the model only needs basic tool-calling. It helps frontier models (token savings) and is required for small local models (they cannot fit large data in the prompt).

Can I share state with rlm-opencode? Yes — both use the same kernel runtime and the same JSONL lake format, so data written by one editor is readable by the other.

Where does state live? ~/.pi/agent/rlm-state/ — kernel snapshots per session, context lake per project (JSONL).

Can I use it with my own Python environment? Yes — set RLM_KERNEL_PYTHON to any interpreter (venv, conda, system).

Is the data sent anywhere? No. Everything runs locally: kernel, lake, and (with a local model) the LLM itself. Your data never leaves your machine.

Publish

This package is published to npm and listed on https://pi.dev/packages (the PI plugin registry).

License

MIT