xp

Autonomous experiment daemon. Point an LLM at any benchmark, it optimizes the metric in a loop.

Built with Effect v4 and Bun.

Install

bun run build   # compiles binary to bin/xp + symlinks to ~/.bun/bin/

Usage

# Start an experiment
xp start optimize-fft \
  --metric latency --unit ms --direction min \
  --benchmark "./bench.sh" \
  --objective "reduce FFT latency" \
  --provider claude

# Monitor
xp status            # current state
xp logs              # daemon output
xp logs -f           # tail daemon output
xp results           # all trial results
xp results --last 5  # last 5 trials

# Steer the agent mid-run
xp steer "try SIMD intrinsics instead of auto-vectorization"

# Stop
xp stop

Commands

| Command | Description | | ------------------ | ----------------------------------------- | | start <name> | Initialize and start an experiment | | stop | Stop the daemon | | status | Show experiment state (--json) | | logs | View daemon log (-f to follow) | | results | Show trial results (--last N, --json) | | steer <guidance> | Send guidance to the running experiment |

start Flags

| Flag | Description | Default | | ------------------ | -------------------------------------------- | -------- | | --metric | Metric name to optimize | required | | --unit | Metric unit | "" | | --direction | min or max | required | | --benchmark | Shell command that emits METRIC name=value | required | | --objective | What the agent should optimize | required | | --provider | claude or codex | claude | | --max-iterations | Budget cap | 50 | | --max-failures | Max consecutive failures | 5 |

Benchmark Contract

The benchmark command must print metrics to stdout in this format:

METRIC latency=42.5
METRIC throughput=1200

One METRIC name=value per line. The --metric flag selects which one to optimize.

How It Works

1. Baseline: runs the benchmark on the current code to establish a starting point
2. Loop: invokes the LLM agent with context (objective, best score, dead ends, user guidance), agent makes changes in a git worktree, benchmark runs, result is kept or reverted
3. Persistence: all events logged to append-only JSONL, crash-safe with two-phase decisions
4. Worktree isolation: experiments run in .xp/worktree/ on an xp/<name> branch — your working directory stays clean

Development

bun run dev -- --help   # run from source
bun run gate            # typecheck + lint + fmt + test + build
bun test                # tests only