loop-infer

A Clojure implementation of Recursive Language Models (RLMs) - a task-agnostic inference paradigm that enables LLMs to handle arbitrarily long contexts by storing data in a sandboxed REPL environment.

Based on the paper "Recursive Language Models" by Zhang, Kraska, and Khattab.

How It Works

Instead of passing large contexts directly to the model, RLMs:

1. Store context in a sandboxed Clojure REPL
2. Let the LLM write code to examine, decompose, and query the context
3. Execute code safely and return results to the LLM
4. Repeat until the LLM produces a final answer

The LLM can also spawn recursive LLM calls for analysis of context chunks.

Installation

Via npm (Recommended)

# Run directly without installation
npx @unravel-tech/rlm-cli -c document.txt -q "Summarize this"

# Or install globally
npm install -g @unravel-tech/rlm-cli
rlm-cli -c document.txt -q "Summarize this"

From Source

Prerequisites:

• Java 21+
• Clojure CLI

git clone https://github.com/unravel-team/loop-infer.git
cd loop-infer

Set your API key:

export OPENAI_API_KEY="sk-..."
# or
export ANTHROPIC_API_KEY="sk-ant-..."

CLI Usage

# Query a file
rlm -c document.txt -q "Summarize the key points"

# Query from stdin
cat large-file.txt | rlm -c - -q "Find all dates mentioned"

# Inline context
rlm -C "The capital of France is Paris." -q "What is the capital of France?"

# Query a directory of data files
rlm -d ./data -e parquet,csv -q "What patterns do you see?"

# Use a different model
rlm -m anthropic/claude-sonnet-4-20250514 -c doc.txt -q "Explain the main argument"

# Generate an HTML execution report
rlm -c doc.txt -q "Summarize" -H report.html

# Verbose output with JSON format
rlm -c doc.txt -q "Summarize" -v -o json

Server Mode

RLM can run as an HTTP server with real-time streaming support:

# Start server on default port 8080
rlm --server

# Start on custom port
rlm --server --port 3000

HTTP API Endpoints

| Endpoint | Method | Description | |----------|--------|-------------| | /health | GET | Health check, returns {"status": "ok", "active-tasks": N} | | /api/query | POST | Synchronous query, returns JSON result | | /api/stream | POST | Streaming query with SSE progress events | | /api/task/:id | GET | Get task status | | /api/task/:id/report | GET | Get HTML execution report | | /api/task/:id/live | GET | Live-updating HTML report page |

Query Example

curl -X POST http://localhost:8080/api/query \
  -H "Content-Type: application/json" \
  -d '{
    "context": "The quick brown fox jumps over the lazy dog.",
    "query": "What animals are mentioned?",
    "model": "openai/gpt-4o",
    "trace": true
  }'

Response:

{
  "answer": "The animals mentioned are a fox and a dog.",
  "costs": {
    "total-cost": 0.00123,
    "root-llm-calls": 2,
    "recursive-llm-calls": 0
  },
  "trace": { ... }
}

Streaming Example

curl -X POST http://localhost:8080/api/stream \
  -H "Content-Type: application/json" \
  -d '{"context": "Hello world", "query": "What is this?"}'

Returns Server-Sent Events (SSE) with real-time progress:

• connected - Task created, returns task ID
• start - Query execution started
• iteration-start - New iteration beginning
• llm-call - Root LLM request/response
• repl-execution - Code execution result
• recursive-llm - Recursive LLM call
• result - Final answer with costs
• done - Execution complete

Live Report

Open http://localhost:8080/api/task/{task-id}/live in a browser to see a real-time updating execution report.

Library Usage

(require '[loop-infer.core :as rlm])

;; Create an RLM instance
(def r (rlm/create-rlm {:model "openai/gpt-4o"
                        :trace true
                        :verbose true}))

;; Query with context
(rlm/query r {:context "Paris is the capital of France."
              :query "What is the capital of France?"})
;; => "Paris"

;; Check costs
(rlm/cost-summary r)
;; => {:total-cost 0.00123 :root-llm-cost 0.001 :recursive-llm-cost 0.00023 ...}

;; Generate HTML report (if trace was enabled)
(rlm/write-html-report r "report.html")

;; Reset for reuse
(rlm/reset-rlm! r)

CLI Options

| Flag | Default | Description | |------|---------|-------------| | -m, --model | openai/gpt-4o | LLM model (provider/model format) | | -r, --recursive-model | (same as --model) | Override model for recursive calls | | -c, --context | - | Context file path (or - for stdin) | | -C, --context-string | - | Inline context string | | -d, --directory | - | Load files from directory as context | | -p, --pattern | - | Glob pattern for directory files | | -e, --extensions | - | Comma-separated file extensions | | -n, --max-files | - | Maximum files to load | | -q, --query | - | Query to answer (required in CLI mode) | | -i, --max-iterations | 20 | Max RLM iterations | | -v, --verbose | false | Enable verbose output | | -o, --output | text | Output format: text, json, edn | | -H, --html-report | - | Generate HTML report to file | | -S, --server | false | Run as HTTP server | | -P, --port | 8080 | Server port | | --host | 0.0.0.0 | Server host to bind | | -k, --api-key | - | API key (or use env vars) |

Supported Models

Any model supported by litellm-clj using provider/model-name format:

• OpenAI: openai/gpt-4o, openai/gpt-4o-mini
• Anthropic: anthropic/claude-sonnet-4-20250514, anthropic/claude-3-5-haiku-20241022

Supported File Types

• Text: txt, md, rst, html, xml, yaml, json, py, js, clj, etc.
• Data: parquet, csv, tsv, json, jsonl, arrow

Building from Source

# Build uberjar
./scripts/build.sh
# or
clojure -X:uberjar

# Run the jar
java -jar loop-infer.jar --help

Publishing to npm

# Build uberjar first
./scripts/build.sh

# Install jdeploy and publish
npm install -g jdeploy
jdeploy publish

Running Tests

clojure -M:test

Architecture

src/loop_infer/
├── core.clj        # Public API and CLI
├── rlm.clj         # Core RLM iteration algorithm
├── repl.clj        # Sandboxed REPL execution
├── prompts.clj     # System prompts for LLMs
├── server.clj      # HTTP server with SSE streaming
├── trace.clj       # Execution tracing and HTML reports
└── file-loader.clj # File and directory loading

See implementation.md for detailed architecture documentation.

License

MIT