stigmergy

The probe is canon. Everything written down is a cache.

A zero-dependency Node CLI (spec) that reads a codebase the way a careful maintainer would — but deterministically, in seconds, with zero tokens — and hands back a tagged report, a graded health score, a set of marked decisions, and a generated docs/ folder. It is two things fused:

1. The Derived-Spec runtime — a documentation format where a spec stores probes (checks the tree answers), never asserted status, so docs cannot rot. (Contract: STANDARD.md.)
2. A semi-deterministic codebase analyzer/documenter — a pipeline of extractors that derive what a codebase is, what's wrong with it, how to redraw its schema, and which indexes to add — leaving the LLM/human only the irreducible choices.

Stigmergy is coordination by marks left in a shared environment that steer the next agent. That is what this is: build / design / audit harnesses (and a human, and an LLM) coordinating through grep-true marks in a shared spec — no central scheduler, no prose handovers.

Zero runtime dependencies. Node ≥ 18.

Install / run

node src/cli.js <command> [dir]      # from a checkout
# or, once linked:  spec <command> [dir]

Every analysis command takes an optional target directory (defaults to cwd) and a --json flag for machine/LLM consumption.

The command surface

The spec runtime (The Derived-Spec Standard §9)

| command | does | |---|---| | spec init [dir] | scaffold a conforming project (SYSTEM / SURFACE / MAP slots, config, pre-commit gate) | | spec lint | referential integrity — every <file>/<sym>/<ref> resolves (fast) | | spec verify | recompute every @status from probes; report cache drift | | spec cursor | print each track's derived <now> | | spec audit [--gate] | the full invariant set; append a health snapshot. --gate exits non-zero on a violation | | spec build | regenerate every .html view from its .xml + the tree (deterministic) |

The analyzer (deterministic, zero-token)

| command | does | |---|---| | spec scan [dir] | stack · config-coherence · design-system + variant bypass · doc-drift · pain signals · tests | | spec map [dir] | import hubs · top utils (the verbs) · the caching layer (the read model) | | spec aggregate [dir] | cluster repetition → proposed primitives (styling) · shared utils (dup functions) · composites (component stacks) | | spec trace [dir] | the object/type layer: schema sin layer (nulls/Json) · type escapes · object drift · the God-model | | spec redraw [dir] | a proposed redrawn schema — deterministic facts + every decision marked // CHOOSE | | spec query [dir] | the query layer: missing indexes (speed edges) · dead relations · never-queried columns | | spec roles [dir] | the architecture loop by invariant: io / wiring / transform / state-long / state-short, found framework-free | | spec strata [dir] | excavate the dead/legacy layer: fossils (priors carried into live), drift (migration deltas), abandoned, schema lineage | | spec graph [--deps] [dir] | an interactive 3D force-graph of the spec — or, with --deps, the raw import web | | spec normalize [dir] | run the whole pipeline → health + tagged report + decision cards + docs/ |

spec normalize — the one command

spec normalize ./my-app

Runs scan → map → aggregate → trace → redraw → query → roles in sequence and emits, into docs/ (or docs/stigmergy/ if a docs folder already exists):

• normalize-report.xml — the report, tagged throughout and logically sequenced, built for an LLM to read.
• decisions.html — the findings as cards, grouped by stage; the coder/agent checks the ones to accept and downloads decisions.json.
• decisions.json — the handoff. The deterministic engine applies the accepted decisions as a first pass with no LLM, then the agent takes the rest.
• DEVSPEC.md — the derived system map and the entry point to diff from.
• DEVSPEC-HISTORY.md — an append-only baseline that normalizes the build going forward (diff each run against the first snapshot).
• knowledge-graph.html — the import web, rendered.
• vendors/<dep>/README.md — one folder per dependency with its docs URL and usage count, so an agent starts at the source instead of searching.

It never clobbers human-edited docs.

Health — under stigmergy's criteria

normalize grades the codebase A–F across: wiring (reachability), architecture (share of files placed in the dataflow loop), type-integrity (the schema sin layer + type escapes), tables (missing indexes / dead relations), constants (magic values living as variables), design-system (variant bypass), tests, and docs (presence

• drift). Each metric shows its evidence; the overall is a transparent weighted score.

The DB anchor — never misfire on any database

Schema discovery does not key on the ORM. It finds the auth landmark — every app has a user / account / person / admin / tenant table — in any recognizable table-definition shape (Prisma model, Drizzle *Table, TypeORM @Entity, Mongoose model, SQL CREATE TABLE), and that locates the DB and identifies the dialect at once. A new ORM only needs a field-parser; the anchor already finds it, because it keys on the entity, not the syntax. (Proven on Prisma and Drizzle.)

The architecture loop — find every framework-dependent piece without framework names

The auth-anchor, generalized to the whole architecture. Every app, in every framework, present or future, is the same dataflow loop:

USER  ⇄  [io]  ⇄  [wiring]  →  [transform]  ⇄  [state-long]   (the DB)
                                            ⇄  [state-short]  (cache / session)

spec roles places every file into that loop by what the code does (returns markup, writes a response, binds a key to a handler, holds a mutable singleton, declares a schema) and where it sits in the import graph — never by a framework's name for it. Framework names (react, express.Router, zustand, @Controller) are tells: they confirm a role and raise confidence, but are never required. So a stack you never coded for — Angular, Svelte, a bespoke runtime, one not invented yet — lands in the same five boxes, because the structural signatures are framework-free. We code for the loop, not the library.

The governing invariant: a role is reported only when structure establishes it or a tell names it — otherwise it's N/A, never guessed. Coverage is always reported, so "found" and "couldn't place" stay distinguishable. On a live Next.js + Drizzle SaaS this reads the io layer (confirmed: react/next), the wiring layer structurally (file-system routing, no framework tell needed), and anchors state-long on schema.ts via the auth landmark — zero config. (Wired into spec normalize as the architecture health metric.)

Code roots — discovered, not assumed

The most common silent failure of a codebase tool is to hardcode src/ and quietly analyze a fraction of the repo. stigmergy discovers the code roots structurally (detectCodeRoots): every top-level directory that holds source, monorepo packages/* children descended, archive/ legacy/ vendor/ examples/ excluded by name (and reported — never silent), dotdirs and build output skipped. The analyzer spans all included roots as one combined import graph, and DB discovery excludes the dead ones so a stale archived schema can't mis-anchor the live database. Pointed at a Next.js app whose src/ (266 files) hid a second engine/ root (272 — the actual engine) and a 738-file archive/, it reads all three correctly and shows the seam between them — src/ is io+wiring, engine/ is the transform layer.

Strata — the dead layer is the fossil record

Excluding archive//legacy/ from the live analysis is right (a dead schema must not mis-anchor the live DB) — but excluding it from everything throws away the system's history. When live code is weird but not broken, the cause is often an older system's prior grafted forward. spec strata reads the temporal dead layers and cross-references them with the live code — deterministically, via the structural hash already computed per function:

• fossils — a live function carrying a shape from the dead layer (≡ an exact lift, ≈ the same logic reshaped/renamed). The first place to look when something is weird-not-broken.
• drift — a name in both strata with a diverged shape: the migration delta (a half-done one is a prime weirdness site).
• abandoned — concepts that live only in the dead layer (context for what's gone).
• schema lineage — data-model tables carried forward vs dropped.

Pointed at a project whose archive/ was a prior version of itself: 0 function fossils but 29 schema tables carried forward and 86 dropped — the data model was inherited though every function was rewritten, and the dropped tables named a whole subsystem the live project is rebuilding. The fossil record explained the present.

LLM-native

Every analysis command emits --json, and the package ships a Claude-Code skill (skill/SKILL.md) — the agent playbook for driving the tool: run normalize, read the tagged report, consume the JSON, apply the deterministic first pass, then reason only about what's left. The whole design is propose-deterministically, choose with judgment — the tool surfaces candidates with evidence; the LLM/human decides.

Self-hosting & acceptance

The runtime re-expresses itself as a conforming spec (spec/STANDARD.xml, spec/BUILDSPEC.xml) and audits its own repo with zero violations; spec build regenerates its views with no diff. That is the "REAL" gate.

The "CHEAP" gate was a pre-registered hostile-tree experiment (KATURA-BASELINE.md): on a 245-model / ~357k-LOC monolith, a fresh agent reached maintainer-level understanding through the derived spec for 80k tokens versus a ~15.4M-token (conservative floor) raw comprehension — an inversion ratio of ~192× (lower bound 77×). Strong pass.

Honest by design

Every extractor is a heuristic over the tree, so its outputs are candidates with evidence, not verdicts — which is the point: the tool proposes, you and the agent decide. Known calibration gaps are tracked in ROADMAP.md (query-layer parsing beyond Prisma, the constants metric on UI/3D apps, the variant check beyond cva, and the apply-engine). When a metric can't be measured on a given stack it says so rather than inventing a number.

Layout

src/        the runtime — one module per primitive + per extractor + cli.js
templates/  what `spec init` lays down
spec/       the package's own specs (self-host): BUILDSPEC.xml, STANDARD.xml (+ views)
skill/      the Claude-Code skill (LLM-native interface)
test/       node:test suites (zero-dep)
STANDARD.md          the normative contract
KATURA-BASELINE.md   the acceptance experiment
ROADMAP.md           the next legs

License

MIT.