Captain

npx captain-dev

Warning: this is an autonomous agent. Run inside your repository, it reads your code, creates, edits, and deletes files, and runs shell commands in that directory. Run it on a committed git tree so every change is reviewable, and review what it did with git diff before you trust it. It asks for explicit consent on first run and refuses a dirty working tree unless forced. The software and the hosted service are provided as-is; see LICENSE, TERMS.md, and PRIVACY.md.

Captain is the file search layer for your AI agents. The CLI is the terminal-native way to set it up: it reads your codebase, gets your data indexed, and tunes retrieval against your own corpus until the numbers hold up.

What it's for

When an agent answers from your documents, the answer is only as good as what retrieval handed it. Captain is the layer that finds the right passages, and the part you can measure.

It holds up on the corpora that break generic RAG: FDA pharma filings, 500-page DOCX files for regulatory finance, chart-heavy research papers. If it works on those, it works on your docs.

It generates evaluation questions grounded in your real corpus, then iterates retrieval settings against them until the score holds up. The configurations it tries come from scaling production file-search workloads since RAG first shipped.

Today it tunes the query side: reranking, how deep the candidate pool goes, and excluding layout noise before ranking. Those are the levers that are free to change and instant to re-test.

Coming soon. Baking structure into the collection itself is the next lever and the larger one: chunk and document metadata so a filter can narrow the search before ranking starts, chunk relations so a related chunk travels with the one that matched (post-ANN graph hydration), and parsing changes, since a 500-page PDF or a scanned form needs a different pipeline than a clean text file. Those changes mutate the collection rather than the query, so they land with an undo record behind them.

Data handling and pricing

The CLI's model calls (generating your evaluation questions, reviewing generated code) are paid for by Captain and cost you nothing. They run zero-data-retention: prompt and completion content is used to serve the request and is not retained or trained on.

Captain stores the indexes you explicitly create. Indexing is billed on usage, starting at $0/mo, full breakdown at captain.dev/pricing. Opt-in PII masking redacts detected PII before embedding or writing to the search index, including OCR-visible text in images; it is configured on the Captain side rather than through this CLI.

Captain Sync keeps S3-compatible buckets continuously indexed through event notifications, with scheduled reconciliation as a fallback. Terraform templates for the bucket-side setup live at runcaptain/captain-sync-templates.

What it does

Reads before it asks. Before anything else, it reads your org (which collections exist, which have documents, which have syncs) and your codebase (existing Captain calls, framework, agent SDK, MCP registrations). What it recommends depends on both. Nothing is written during this pass.

Gets your data in. Continuous sync for S3, R2, Supabase, Backblaze and GCS. One-time indexing for Google Drive, SharePoint, OneDrive, Dropbox, Azure, URLs, YouTube and local files, with an optional scheduled re-index you own for the sources that have no sync yet.

Measures retrieval. Builds a map of your corpus, generates an evaluation set of questions grounded in real chunks, and scores retrieval on recall@1, recall@3, recall@10, MRR and latency. Results are a 0-100 score and a letter grade.

Tunes until the score holds up. Runs a baseline, then iterates query configuration (rerank, candidate pool depth, excluded layout noise) until the score plateaus or hits target. It reports what each configuration scored, so the choice is auditable rather than asserted.

Coming soon: a self-tuning agent. Today the tuning loop tries a fixed set of query configurations. The next version runs an agent that reads the per-question results, forms a hypothesis about why specific questions failed, changes the setup, and re-measures, repeating until the score clears a target or a budget you set runs out. It will be opt-in per run and will ask before it touches anything.

Wires itself into your code. An API route, a retrieval tool for your agent, a file-search UI, or MCP registration for your editor.

Requirements

A Captain API key. Get one at captain.dev/api-keys.

Captain reads --api-key, then $CAPTAIN_API_KEY, then .env.local, then prompts. It only ever reads the Captain variables out of a dotenv file and never loads the rest of it into the process.

Commands

npx captain-dev

Just npx captain-dev runs it, no install step. npm i -g captain-dev installs the captain binary if you'd rather have it on PATH permanently.

Deterministic, machine-readable commands, all supporting --json:

captain login [--api-key <key>]         # verify a key with one real call, save it for every future run
captain query <collection> <text> [--limit n] [--rerank]
captain collections
captain sync [--collection <name>]      # lists syncs (optionally one collection's), offers to create one
captain jobs <job_id> [--wait] [--timeout <seconds>]  # --wait polls to completed/failed
captain index <path> --collection <name>  # local file or directory, non-recursive
captain index --provider <gdrive|sharepoint|onedrive|dropbox|azure> --collection <name> \
  [--scope root|directory|file] [--path <path>] [--file-id <id>] [credential flags]
captain index --provider <youtube|url> <url> --collection <name>   # one page or video
captain index --provider text --text <content> --collection <name> # or pipe content on stdin
captain index --provider <s3|gcs|r2|backblaze|supabase> --collection <name> --bucket <bucket> \
  [provider credential flags] [--prefix <prefix>]   # per-run bucket index: credentials each run, nothing stored
captain scan                            # read-only report on this folder; no key needed
captain sync create --provider <s3|gcs|r2|backblaze|supabase> --collection <name> --bucket <bucket> \
  [provider credential flags, e.g. --region --access-key-id --secret-access-key for s3] \
  [--prefix <prefix>] [--processing-type basic|advanced] [--deletion-policy mirror|archive|ignore] \
  [--sync-interval-minutes <n>] [--access-key-id <id> --secret-access-key <secret>] [--confirm]
captain sync create --provider <p> --collection <name> --bucket <bucket> ... --one-time --confirm
                                        # one-time sync: access stored on Captain, indexes now, no schedule; re-index with sync reconcile
captain sync get <sync_id>              # one sync's full state: schedule, last error
captain sync delete <sync_id>           # remove the connector, keep the documents
captain sync pause <sync_id>
captain sync resume <sync_id>
captain sync reconcile <sync_id>        # run a reconcile pass now
captain tune <collection> [--yes] [--gateway-url <url>]  # self-tuning agent; see below
captain promote <collection> <development|staging|production>  # move without reindexing
captain collections create <name>       # idempotent: an existing name is reported, not an error
captain undo [run-id]                   # revert the files a Captain run wrote here, from the .captain/ write manifest
captain telemetry [--enable|--no-enable]  # show or set anonymous usage reporting; CAPTAIN_NO_TELEMETRY=1 always wins
captain completion zsh                  # tab completion; also bash, fish

Tab completion covers every command, subcommand and flag above, plus live values where they exist: collection names after --collection (and as the first positional of query and promote), and real sync ids after sync get|delete|pause|resume|reconcile. The printed script delegates to a hidden captain __complete call, so completions update with the CLI. The comment at the top of each script says where to install it.

captain tune <collection> runs the self-tuning agent: it scores the collection's eval question set (.captain/evals/questions.jsonl, generated by a prior Optimize pass), forms a hypothesis from the per-query misses, applies a query-config change, and re-scores until recall@10 meets target or the budget runs out. Nothing runs until the consent gate passes: the agent reads code, edits files, and runs shell commands in the current directory, and the first run says so and asks (--yes accepts it non-interactively). The LLM gateway is $CAPTAIN_AGENT_GATEWAY_URL, then --gateway-url, then the pointer the Captain API serves at /agent/config (so the gateway can move or fail over without a CLI release), then the baked-in production default. With --json, progress streams to stderr and one envelope lands on stdout. In the interactive TUI, populated collections offer a "Tune retrieval (agent)" action behind the same consent gate.

For coding agents

Every command is non-interactive with --json: one JSON envelope on stdout (ok, error, fix, next_actions), progress and warnings on stderr, and meaningful exit codes (0 ok, 2 auth, 3 validation). Nothing prompts when stdout is not a TTY. Headless tune is captain tune <collection> --yes --json; --yes is the consent for the agent to read and modify files in the working directory, so pass it only when your own user has approved that.

The --provider form of index runs a one-time index against a provider with no continuous sync -- re-run it to pick up new or changed files. Each provider's credential fields are passed as flags (e.g. --dropbox-access-token, --tenant-id --client-id --client-secret --site-url for SharePoint), and each also falls back to an environment variable named CAPTAIN_<PROVIDER>_<FIELD> (e.g. CAPTAIN_DROPBOX_DROPBOX_ACCESS_TOKEN). In an interactive terminal, any credential still missing after flags and environment variables is prompted for; in a non-interactive run (--json, or no TTY, such as CI) missing credentials are a clear error instead of a hang.

sync create sets up continuous sync non-interactively for any of the five sync providers (--provider defaults to s3). Each provider's fields resolve flag, then CAPTAIN_<PROVIDER>_<FIELD>, then the same credential discovery the interactive flow uses (AWS_*, B2_*, S3_PROTOCOL_*, CLOUDFLARE_ACCOUNT_ID, GOOGLE_APPLICATION_CREDENTIALS), then a prompt in interactive sessions only. captain sync create --help lists the fields per provider.

There are two ways to index a bucket without a schedule, and they differ in where the credentials live:

• captain sync create --provider <p> ... --one-time --confirm creates a one-time sync: a real sync connector with your access stored on Captain and no schedule (sync_interval_minutes is null, which the API treats as manual reconcile). It indexes the bucket on creation and again only when you run captain sync reconcile <sync_id> (or "Reconcile now" under Manage sync in the TUI). Use it when you do not want to enter credentials again later. It is the headless form of the interactive flow's "One-time" row on the deletion-policy screen, follows confirm-then-mutate, and refuses a duplicate like any other sync. --sync-interval-minutes is rejected with it.
• captain index --provider <s3|gcs|r2|backblaze|supabase> ... is the per-run index: credentials are supplied on each run and stored nowhere, one request goes to /index/{provider}, and the result is an index job like the other index --provider values. Same required fields, flags and CAPTAIN_<PROVIDER>_<FIELD> fallback as sync create. Re-run it to pick up changes.

For coding agents

Add --json to any command -- or just run with no TTY at all, which is the normal case for CI or an agent harness -- and every prompt is skipped in favor of exactly one JSON value on stdout:

{ "ok": true, "result": { /* command-specific */ }, "next_actions": ["captain query docs \"...\" --json"], "version": "0.4.0" }

Human-readable progress and warnings go to stderr, so stdout always parses cleanly as that one JSON value. next_actions is the field worth reading first: the exact next captain ... command to run, not prose to reconstruct flags from.

Exit codes are meaningful and stable: 0 ok, 1 error, 2 auth (missing or bad API key / provider credentials), 3 validation (bad input, caught before any network call), 4 confirmation required.

Exit 4 is the whole answer to "how does a command that mutates something work with no human to answer a y/n prompt": captain sync create never creates anything on its first call. Without --confirm it validates every input, resolves credentials, and returns exit 4 with the exact request it would send as result.request -- nothing happens. Re-run the identical command with --confirm (the exact string is in next_actions) to actually create it. Any credential in that preview is masked, never echoed in full, even though nothing was created yet -- this JSON is exactly the kind of output that ends up in an agent's transcript or a CI log.

captain --help --json returns the full command tree as structured data -- every command, its flags, and the envelope/exit-code contract above -- so an agent can discover the whole surface without parsing this file.

captain sync create --provider s3 --collection docs --bucket my-bucket --region us-east-1 --json
  # -> exit 4, result.request is the proposed sync body, nothing created
captain sync create --provider s3 --collection docs --bucket my-bucket --region us-east-1 --confirm --json
  # -> exit 0, result is the created sync connector
captain query docs "what is our refund policy" --json
  # -> exit 0, result.results is the ranked list

The interactive wizard (bare captain at a real terminal, built on Ink) is for humans and isn't meant to be driven by scripted keystrokes -- it's a full-screen, raw-mode terminal renderer, and forcing an agent to fake keypresses into it is fighting the wrong layer. Machine access is this --json contract, or the Captain MCP server for an agent operating Captain directly rather than through a shell.

Telemetry

Captain collects anonymous usage telemetry to improve the CLI. Telemetry is opt-in and disabled by default.

What is collected

When enabled, Captain posts a small JSON event to the Captain API's feedback endpoint (POST /feedback on the configured --base-url). Today the only event is the one fired when you answer yes to "File this as a request?" in the interactive UI. Each event carries:

• The action name (e.g. do-something-else) and where in the UI it happened
• Success or failure status, and the error class if it failed
• Client version and a timestamp
• For a filed request only: the text you typed, prefixed with the name of the collection it was about

What is never sent:

• Your API key
• Query text
• File contents
• Collection names, except inside a request you explicitly chose to file
• Any personally identifiable information

Opting in or out

Enable telemetry:

captain telemetry --enable

Disable telemetry:

captain telemetry --no-enable

You can also permanently disable telemetry by setting an environment variable:

export CAPTAIN_NO_TELEMETRY=1

Or respect the DO_NOT_TRACK=1 convention.

Check current status:

captain telemetry

Notes on measurement

The score is a weighted composite of four things: whether the right document is found at all, whether it is found in the top three, whether it is ranked first, and on multi-hop questions whether the set is ordered well.

recall@3 is weighted heavily on purpose. An agent reads the first few results, so a pipeline can look excellent at recall@10 while starving the thing actually consuming it.

F1 is deliberately not reported. With one relevant document per question, precision@10 cannot exceed 0.1, so F1@10 caps near 0.18 on flawless retrieval. It would render a perfect pipeline as a failing grade.

nDCG is only reported when the evaluation set contains multi-hop questions. With a single relevant item it reduces to a monotone function of rank, which is information MRR already carries.

Working directory

Captain writes .captain/ into your repo, and it is meant to be committed:

.captain/
  config.json              collection, environment, chosen query config
  corpus-map.md            structural map of your corpus
  evals/
    questions.jsonl        evaluation set with ground truth
    runs/                  per-run metrics and summaries
  logs/                    gitignored, raw numbers for if a human needs to debug

The question set is the expensive artifact. Keeping it in version control means reruns are cheap, runs are comparable over time, and your team shares one evaluation baseline.

Development

npm install
npm test
npm run typecheck
npm run dev

The Captain API gets upgraded very frequently. npm run check-schema-drift fetches the live spec at docs.captain.dev/openapi.json and checks that the fields src/api/types.ts, sync.ts and onetime.ts depend on still exist under the names this CLI expects -- run it before a release. npm run sync-openapi pulls the full generated types (src/api/openapi.generated.ts, gitignored) for investigating a failure.

License

MIT