@dev.fast/code-graph

Headless SCIP indexing for TypeScript, JavaScript/JSX, Go, Rust, and Python. This document describes the unpublished 0.2.0 package. The package stores its graph in SQLite and exposes refresh, daemon, editor, and call-tree interfaces.

Quick start

TypeScript/JavaScript and Python use package-owned SCIP indexers. Python also requires Python 3.10+ plus pip3 or pip from the active environment. Selected Go projects use scip-go plus go; selected Rust projects use a rust-analyzer build with the scip subcommand plus cargo and rustc. rust-src is optional, but its presence and identity are fingerprinted because it changes Rust output. Missing Go/Rust tools or required host prerequisites are reported once per discovered family and skipped by default. A failed project is also isolated from other projects in its language family, so every successfully completed project can still be published. Pass --require-indexers for strict CI behavior: missing prerequisites and failed projects both fail closed before the graph is published.

Each language planner separates discovered candidate files from documents the indexer is required to emit. Emitted candidate documents are ingested, while strict completeness checks only the language's active project universe. Go derives that universe from go list in the indexer environment; Rust requires Cargo target roots and accepts emitted crate-module documents. This keeps platform/build-tagged Go files and Rust files outside the crate module graph from becoming false incomplete-output failures. Plans identify an authoritative or coverage context and may add context-local environment overrides. Context precedence is deterministic: authoritative output wins, followed by coverage contexts in lexical label order.

Candidate files legitimately outside every required indexer context still appear in the graph. Tree-sitter extracts their standalone file, declaration, import, and structural facts without creating cross-file semantic edges. Every file records one of semantic, semantic-coverage-context, syntax-only, or none; refresh and per-indexer summaries report counts for each status.

Rust's authoritative context asks rust-analyzer for all Cargo features and targets through a generated scratch config. If that widened configuration is invalid for a workspace (for example, mutually exclusive features), indexing retries once with rust-analyzer's default Cargo settings and records the downgrade on the project summary. Unreferenced Rust files remain syntax-only.

Go adds coverage contexts only for target files omitted by the authoritative build context. Incremental review extraction uses the changed files as targets; full extraction considers all selected Go files but creates at most indexer.maxCoverageContexts coverage plans (default 2). Planning evaluates build expressions and filename constraints against Linux, Darwin, and Windows on amd64 and arm64, adding literal positive build tags through GOFLAGS. Unmatched or over-cap files remain syntax-only.

Authoritative documents and symbols win deterministic merge conflicts. Coverage documents only fill missing files; their declarations retain the context label, and references resolve in that context before falling back to authoritative symbols. File nodes and persisted SCIP symbol rows carry context provenance so incremental refresh can re-merge deterministically.

pnpm --filter @dev.fast/code-graph refresh
pnpm --filter @dev.fast/code-graph refresh -- --root /path/to/repo --languages python,go
pnpm --filter @dev.fast/code-graph refresh -- --languages all --require-indexers

The exact supported language list is typescript, javascript, go, rust, and python. --languages all enables discovery for all five. An indexer is probed and started only when matching selected files exist. Non-zero probes, indexer crashes, malformed output, empty output, and project timeouts are recorded as typed project failures in normal mode. Output from a failed process is never ingested. Zero declarations from nonempty successfully indexed sources remains a hard correctness failure. Individually unresolved relationships are counted but do not fail extraction.

An explicit family command is authoritative and never falls back to PATH. The public @dev.fast/code-graph/indexers entry point exposes source-driven repository inspection and tool status APIs.

Configuration

Pass a JSON file with --config, or set CODE_GRAPH_CONFIG to a file path or inline JSON:

{
  "indexer": {
    "projectConcurrency": 4,
    "ignorePaths": ["fixtures/archive", ".cache/generated"],
    "families": {
      "go": {
        "command": "scip-go",
        "args": [],
        "env": {}
      }
    }
  }
}

Configuration is strict. The root accepts only indexer; indexer accepts only projectConcurrency, ignorePaths, and families; each family accepts only command, args, and env. The default project concurrency is capped at four.

indexer.ignorePaths contains repository-relative paths. Each path excludes itself and every descendant from discovery, incremental scans, refresh watching, and staleness checks. Paths are normalized, deduplicated, and sorted; absolute paths, repository-root or escaping paths, globs, and negation are rejected. Built-in exclusions for .dev/code-graphs and .dev/software-maps always apply. CODE_GRAPH_CONFIG remains the sole configuration entry point.

Indexing model

One canonical registry owns each indexer family's launch, project identity, incremental scope, project markers, source extensions, and language profiles. TypeScript and JavaScript share the TypeScript family.

Built-in indexers implement the internal versioned ScipFamilyPlugin contract under src/scip/families/ and are installed by the static tuple in src/scip/installed.ts. A family plugin owns its language profiles, project planning, launch identity, and value-only ingest refinements. Tree-sitter syntax enrichment is an optional language capability, so a valid SCIP indexer can provide declarations, symbol ownership, direct SCIP relationships, semantic coverage, and metrics without syntax-derived edges. This is a contributor boundary, not a runtime third-party plugin API or package export.

To add another built-in family, define its versioned family plugin and language profiles under src/scip/families/, including source extensions, project markers/planning, probe behavior, and prerequisites. Register the plugin in the static catalog at src/scip/installed.ts, then add unit, semantic fixture, packed package, and external-indexer coverage. Dynamic third-party package loading is intentionally out of scope.

Indexing runs by TypeScript config group, Go module, Cargo workspace, or Python project/environment. Every index is written to scratch space and streamed without materializing the full protobuf. SCIP definitions establish stable graph identities, then bounded Tree-sitter syntax facts join imports, calls, callbacks, hierarchy, type annotations, and state access to SCIP occurrences. Named local variables are retained from declaration syntax; parameters, anonymous outline frames, and object-literal breadcrumb entries are not used as semantic authority.

The semantic edge set is IMPORTS, IMPORTS_SYMBOL, REEXPORTS, CALLS, INSTANTIATES, COMPOSES, EXTENDS, IMPLEMENTS, RETURNS_TYPE, PARAM_TYPE, PROPERTY_TYPE, REGISTERS_CALLBACK, READS, and WRITES. Module edges may target an indexed file or ExternalPackage; all other semantic targets must resolve to one unambiguous declaration. Named callbacks require stable callable declarations, and anonymous closures are omitted. Unresolved targets are counted; the indexer never invents internal file nodes or dangling edges for them.

Only the JavaScript, TypeScript, TSX, Go, Rust, and Python WASM grammars are referenced. The production dependency tree contains no native tree-sitter, native grammar package, or node-gyp dependency.

SQLite and incremental refresh

Refresh writes a complete replacement database under .code-graph/work and atomically publishes .code-graph/state/graph.sqlite. Incremental refresh copies unchanged rows and patches affected source ownership. It reads previous inbound relationships and reindexes unchanged reverse dependents when their targets move or disappear.

Schema v4 persists non-local symbol identity in scip_symbols(family, project, symbol, node_id, source_file). Compatibility metadata includes extractor and schema versions, enabled languages, syntax and configuration fingerprints, SCIP tool and aggregate indexer fingerprints, and indexer usage. indexer_tools records source-driven availability, command, version, prerequisite versions, and discovered-file counts. The v4 indexer fingerprint also includes the operating system and architecture. Adding, removing, or changing a tool or prerequisite, or moving the graph between platforms, forces a full rebuild without changing schema v4.

graph_completeness and project_failures make partial graphs explicit. An incomplete graph is usable for the current session but cannot seed incremental refresh and is retried on the next session. Strict mode preserves the last good database and rejects the incomplete replacement.

Structural folding is intentionally absent from the persisted graph, snapshot, editor, and call-tree state. Compact source views use explicit elisions in the view specification.

sqlite3 .code-graph/state/graph.sqlite \
  "SELECT label, count(*) FROM nodes GROUP BY label ORDER BY label;"

sqlite3 .code-graph/state/graph.sqlite \
  "SELECT type, count(*) FROM edges GROUP BY type ORDER BY type;"

Verification

The default unit suite and the package-owned TypeScript/JavaScript/Python fixtures are offline. Go/Rust checks and the network corpus benchmark are explicit:

pnpm --filter @dev.fast/code-graph typecheck
pnpm --filter @dev.fast/code-graph test:unit
pnpm --filter @dev.fast/code-graph test:fixtures
pnpm --filter @dev.fast/code-graph test:semantic
pnpm --filter @dev.fast/code-graph test:scip-parity
pnpm --filter @dev.fast/code-graph test:scip-incremental
pnpm --filter @dev.fast/code-graph test:pack
pnpm --filter @dev.fast/code-graph test:external-indexers
pnpm --filter @dev.fast/code-graph benchmark:indexing-corpus -- --verify-baseline
pnpm --filter @dev.fast/code-graph verify:scip-only
pnpm --filter @dev.fast/code-graph verify:plugin-boundaries

benchmark:indexing-corpus is an opt-in testing and measurement tool, not a release or publish gate. Pass --verify-baseline to compare pinned repositories with their committed structural and semantic expectations, or --measure --repository <id> to print a candidate baseline. Like test:semantic -- --measure, it never rewrites committed fixtures. Corpus mirrors, archives, and extracted sources are cached under ~/.cache/dev-fast/indexing-corpus.

Editor and daemon

The daemon, SQLite-backed editor, snapshot resolvers, and call-tree interfaces consume declaration and relationship graph records:

pnpm --filter @dev.fast/code-graph daemon
pnpm --filter @dev.fast/code-graph editor
pnpm --filter @dev.fast/code-graph call-tree