repo-context

repo-context gives agents and developer tools focused, verifiable answers about a codebase—without sending a broad repository search into context.

[!CAUTION] Evidence confidence: low — experimental. The deterministic commands and safety contracts are covered by automated tests, but the current real-repository performance evidence is too small and insufficiently reproducible to support general claims about retrieval accuracy or end-to-end token savings.

It uses a small repository map to answer three practical questions:

• Where does a product concept live?
• What is affected by these changes?
• Which tests and verification commands should run?

Install

npm install --save-dev @hamedb89/repo-context

Or run it without installing:

npx @hamedb89/repo-context map check --json

Requires Node.js 20+ and Git. map check also requires ripgrep; bench requires hyperfine.

For development, this repository pins Bun with mise. Run mise run test for the fast suite, mise run check for Bun plus Node compatibility, and mise run orient to dogfood the local CLI.

Quick start

Create repo-context.json in the repository root:

{
  "schemaVersion": 1,
  "mapPath": "docs/architecture/repo-map.json",
  "workspace": {
    "roots": ["apps", "packages"],
    "localPackagePrefixes": ["@acme/"],
    "nativeBoundaries": []
  },
  "markers": {
    "prefix": "acme"
  },
  "source": {
    "ignoredPaths": [".repo-context/benchmarks/baseline.json"]
  },
  "search": {
    "profiles": {
      "default": {
        "roots": ["apps", "packages", "docs"],
        "ignoredPaths": ["generated"],
        "limit": 40
      }
    }
  },
  "commands": {
    "mapInspect": "repo-context map inspect {domain} --json",
    "mapCheck": "repo-context map check --json",
    "fallbackSearchRoots": ["apps", "packages", "docs"],
    "fallbackTest": "pnpm test",
    "testsByDomain": {
      "api": ["pnpm --filter @acme/api test"]
    },
    "verificationRules": [
      {
        "domains": ["contracts"],
        "commands": ["pnpm verify:contracts"]
      }
    ]
  },
  "benchmark": {
    "artifactDirectory": ".repo-context/benchmarks",
    "baselinePath": ".repo-context/benchmarks/baseline.json",
    "scenarios": []
  }
}

Then add a repository map at docs/architecture/repo-map.json (or change mapPath). The versioned JSON schemas in schema/ describe both files.

Mark important source files with the configured prefix:

// @acme:domain api
// @acme:role application-entry
// @acme:flow http-request

Ask focused questions:

repo-context orient --json
repo-context where authentication --summary --json
repo-context agent instructions --json
repo-context search authentication --profile default --json
repo-context context authentication --budget 800 --json
repo-context inspect file packages/api/src/app.ts --json
repo-context impact --changed --summary --json
repo-context test affected --changed --summary --json
repo-context map check --json
repo-context bench
repo-context bench ab authentication --budget 800 --json

The JSON output includes an estimated response-token size (UTF-8 bytes / 4). It is a context-size guide, not provider billing.

Stable JSON receipts

Every JSON response has a versioned envelope: schemaVersion, status, command, and command-specific data. Discovery receipts additionally report source state, confidence, evidence, truncation, and next actions. Consumers should branch on schemaVersion and status before reading data.

orient --json is the canonical first call. It reports the mapped domains, workspace boundaries, supported commands, and next bounded queries. agent instructions --json remains a compact compatibility-oriented bootstrap command for consuming skills.

orient --summary --json is the compact bootstrap receipt. It validates map health before reporting verified confidence and contains only domain IDs, profiles, capabilities, source state, and configured follow-up commands.

Workspaces and semantic domains

Domains may cover more than one workspace. Legacy maps can keep boundary and dependsOn; new maps use boundaries and optional workspaces for the dependency graph:

{
  "version": 1,
  "workspaces": [
    { "id": "planner-core", "path": "packages/planner-core", "domain": "planner" },
    { "id": "web", "path": "apps/web", "domain": "web", "dependsOn": ["planner-core"] }
  ],
  "domains": [
    { "id": "planner", "description": "Planning.", "boundaries": ["packages/planner-core"], "landmarks": [] },
    { "id": "web", "description": "Web.", "boundaries": ["apps/web"], "landmarks": [] }
  ]
}

Impact traverses workspace dependencies first, then returns both workspace and domain in each affected result.

map check --json prints its validation receipt in all cases and exits with code 2 when the map is invalid, making it suitable for CI guards.

Add --measure-savings to report two baselines. configured-search-roots-v1 is the legacy whole-search-roots comparison. For successful context commands, targeted-rg-v1 is an explicitly modeled workflow of two bounded rg searches and up to three complete file reads. It counts the full temporary search output, but persists only aggregate metadata. Neither baseline is provider billing or observed end-to-end agent use. Add --record-usage to append the privacy-safe aggregate to .repo-context/usage.jsonl (configurable with usage.artifactPath).

repo-context orient --json --measure-savings
repo-context impact --changed --json --record-usage

Use usage.baselineProfile to choose which configured search profile defines both baselines. The modeled baseline is unavailable for non-context commands because no query is inferred.

Context packs and local A/B benchmarks

context is a read-only retrieval prototype for repositories that may not yet have a repository map. It groups bounded search evidence by file and returns small source windows within a token budget:

repo-context --root /path/to/repository context authentication --budget 800 --json

Repositories with an authored map can opt into a bounded semantic Mermaid map alongside source evidence. The local cache retains typed facts and rendered views, then revalidates only map and workspace-manifest evidence. See semantic context cache.

bench ab compares that context pack with an exact-search receipt plus whole files. Without gold paths it opens the first search matches; with --expect or an evaluation suite it opens the known gold files, making the result an oracle-assisted context-size comparison rather than a measured user workflow. It does not include source snippets in the benchmark result.

repo-context --root /path/to/clean-snapshot \
  bench ab authentication --budget 800 --baseline-files 5 --json

The A/B command rejects dirty repositories by default. Use a disposable clean clone or worktree for reproducible measurements. --allow-dirty is available only for exploratory local runs.

Evaluation suites

Use an evaluation suite to test positive, negative, ambiguous, and safety cases without storing raw source in the result. Keep suites for private repositories outside this project, because they can contain private task wording and expected paths.

{
  "version": 1,
  "targets": [{
    "id": "example",
    "root": "/path/to/clean-snapshot",
    "cases": [
      {
        "id": "known-auth-flow",
        "kind": "positive",
        "query": "authentication callback",
        "expectedPaths": ["apps/api/src/auth/callback.ts"],
        "expectedTests": ["apps/api/src/auth/callback.test.ts"],
        "budget": 800
      },
      {
        "id": "unrelated-term",
        "kind": "negative",
        "query": "this product does not exist",
        "budget": 800
      }
    ]
  }]
}

Run it with:

repo-context bench suite --suite /private/path/evaluation-suite.json --json

The suite compares exact, ranked rg, repository-map-assisted, fixed gated-map, and repo-calibrated retrieval using the same cases and budget. It reports recall at three stages—discovered candidates, gate-selected candidates, and the final budgeted pack—so a miss can be attributed to discovery, ranking/gating, or context budgeting. See the experiment protocol and current evidence.

Experimental evidence — low confidence

This label applies to the product-effect claims, not to whether the commands run:

| Area | Current confidence | | --- | --- | | Deterministic command behavior and output contracts | Tested | | Bounded packing and abstention mechanics | Tested | | Retrieval quality across real repositories | Low; experimental | | Repository-specific calibration improvement | Low; inconclusive | | End-to-end agent token savings | Not yet measured |

An audit found that the 2026-07-28 runs did not retain a machine-readable result, suite hash, or snapshot fingerprints. The private suite and disposable snapshots were removed as intended, so the documented aggregates cannot now be independently reproduced or used to explain the earlier 0.50 versus later 0.40 result.

A later source audit found an additional defect in the Git-history calibration corpus: it searched HEAD for tasks derived from earlier commits and split cases by hashed ID, not time. That allows completed changes to leak vocabulary into their own answer files and mixes later work into training. The historical calibration figures are therefore contaminated and must not be used as evidence that calibration improves retrieval.

The old runners also used “positive recall” for different calculations: mean path recall, any-path hit rate, and thresholded case success. These historical scores are therefore recorded but not treated as directly comparable:

| Historical run | Reported result | Audited status | | --- | --- | --- | | Initial fixed gated-map suite | 0.70 positive score; 1.00 negative abstention and safety | Development evidence; not independent confirmation | | Earlier calibration holdout | 0.50 calibrated/fallback versus 0.30 fixed | Contaminated Git-history fitting; metric and snapshot identity were not retained | | Placebo falsification rerun | 4/10 calibrated positive cases passed versus 3/10 fixed and 3/10 placebo | Policy fitting was contaminated; raw result is also statistically inconclusive |

The old falsification runner returned survives-this-test because one additional success cleared its fixed 0.05 margin. That rule was too permissive. A paired one-sided sign test gives no better than p = 0.50 for a one-case net advantage, so the corrected interpretation is inconclusive.

The aggregate token observations are still useful, with a narrower meaning:

| Policy | Mean reference tokens | Successful-case reduction | Mean / max byte-estimate error | | --- | ---: | ---: | ---: | | Fixed gated map | 479 | 84.31% | 4.84% / 14.33% | | Real repo calibration | 558 | 63.91% | 4.76% / 14.33% | | Shuffled-label fallback | 479 | 84.31% | 4.84% / 14.33% |

“Reduction” compares a successful bounded pack with an exact-search receipt plus the complete, already-known gold files. It demonstrates that small source slices can be much smaller than reading whole relevant files. It does not measure a real end-to-end search workflow, retries after misses, or total agent token usage.

The defensible claim is therefore: when retrieval succeeds, repo-context can produce substantially smaller source context than reading whole known-relevant files. Repository-specific calibration remains an unproven hypothesis.

See retrieval experiments and the calibration evidence audit for details.

To challenge calibration itself, use the same held-out suite with the falsification runner:

repo-context bench falsify --suite /private/path/evaluation-suite.json \
  --max-cases 40 --record-evidence --json

It evaluates fixed gating, real calibration, and a placebo policy trained on non-overlapping rotated labels. The current runner separates path recall from case success, requires explicit forbidden-path patterns, checks safety and abstention per target, adds a paired significance check, and emits privacy-safe suite and snapshot fingerprints. Its verdict can be falsified, inconclusive, or survives-this-test; the last result is still evidence, not proof.

--record-evidence writes the source-free aggregate result beneath the configured benchmark artifact directory (ignored by default), so the run can be audited without committing the private suite.

Positive cases require expectedPaths; negative cases pass only when the context pack abstains, while safety cases verify configured forbidden-path patterns. The suite compares the fast bytes÷4 estimate with the local o200k_base reference tokenizer. This reference is useful for calibration, but is not provider billing or a guarantee for every model.

Repo-local calibration

calibrate generates local cases from explicitly recorded useful files, repository-map concepts, recent Git commit subjects, and synthetic negatives. It collects rg evidence once per case, evaluates a bounded grid of deterministic ranking and confidence policies in memory, and adopts a policy only when it clears both training and held-out recall and abstention thresholds.

repo-context calibrate --max-cases 40 --json
repo-context calibrate --if-stale --json
repo-context calibrate --if-stale --max-age-hours 12 --json
repo-context cache clean --json

After completing a real task, an agent or developer can record which files were actually useful:

repo-context feedback record \
  --query "where is authentication callback handled" \
  --useful apps/api/src/auth/callback.ts \
  --json

Feedback queries and paths stay in a mode-0600 local file, defaulting to .git/repo-context/feedback.jsonl in Git repositories. They are not returned by the command or written to the calibration cache. Recording feedback changes the repository fingerprint, so the next stale-check recalibrates with the new evidence.

No source, commit subject, query, or matched path is written to the calibration cache. The cache contains the chosen numeric policy, repository fingerprint, aggregate metrics, and timestamp. In Git repositories it defaults to .git/repo-context/calibration.json, outside the tracked worktree.

An adopted policy is automatically used by context. Explicit --strategy continues to override it:

repo-context context "authentication callback" --json
repo-context context "authentication callback" --strategy exact --json

To force the next automatic or explicit calibration to start fresh, remove the local calibration artifact:

repo-context cache clean --json

This clears only the calibration cache; recorded feedback is retained.

Calibration is not run from a package-install script. To enable bounded automatic recalibration on the first context call and after the cache becomes stale, opt in per repository:

{
  "calibration": {
    "auto": true,
    "feedbackPath": null,
    "maximumAgeHours": 24,
    "maximumCases": 40,
    "minimumPositiveRecall": 0.8,
    "minimumNegativeAbstention": 0.9
  }
}

If the repository cannot provide both positive and negative held-out cases, or the winner misses either threshold, the tool records the result but retains the safe default instead of adopting an unvalidated policy.

See the repo-local calibration evidence table for the latest fixed-versus-learned holdout results and current adoption decision.

Configurable runners

Verification commands can use the {runner} placeholder. Configure commands.runner per repository—for example "npm", "bun", "pnpm", or "mise run"—and keep test and verification rules portable:

{
  "commands": {
    "runner": "mise run",
    "fallbackTest": "{runner} test",
    "verificationRules": [
      { "domains": ["api"], "commands": ["{runner} check"] }
    ]
  }
}

Use testsByWorkspace when an affected workspace needs its own test command. Workspace commands take precedence over testsByDomain and receive {workspace}, {domain}, and {path} template values.

Use commands.invocation to make every returned repository-context command pass through a repository façade, such as "./bin/faaast context".

Skill commands

Print a Codex skill that uses the configured invocation, or install it to an explicit directory:

repo-context skill print --format codex
repo-context skill install --format codex --path .codex/skills

Installation refuses to overwrite an existing skill unless --force is supplied.

Agent skills

Use repo-context orient --json as the first repository-specific lookup in a consuming agent skill. It returns a compact, versioned workflow rather than the complete repository map, including the appropriate follow-up commands for concepts, source files, changes, and verification.

The package includes a repo-context skill at skills/repo-context/SKILL.md that follows this workflow. It is intentionally small: repository state remains in the command output so it is current for the checked source revision.

Library API

The CLI is a thin wrapper around the ESM library:

import {
  loadRepoContext,
  whereRepositoryConcept,
} from '@hamedb89/repo-context'

const context = await loadRepoContext(process.cwd())
const result = await whereRepositoryConcept(context, 'authentication')

License

MIT