Arachne

Interactive architecture intelligence for data systems. Map agents, tables, reads, writes, schedules, and architectural risks in a deterministic, searchable report.

Arachne is an interactive architecture explorer. Its static-analysis engine turns source code into a navigable architecture map, a health report, and a versioned stream of structural changes that can be reviewed by people or gated in CI.

Named for the Lydian weaver who out-wove Athena and was turned into a spider to weave forever.

Arachne weaves the codebase into a web and lights up the threads that do not look as they should. It is the deterministic counterpart to the "Snowballs vs. Spiderwebs" idea: a snowball grows by quantity and loses legibility; a web grows by connection, and — like a spider feeling vibration across every strand — you can read its whole shape from any point.

What it does

Arachne's engine is deterministic static analysis (no LLM, no API key, runs in CI). It is plain JavaScript (one zero-dependency .mjs) — nothing to compile or install.

1. Walks the chosen roots and reads every TS/JS and Python file.
2. Extracts the data-flow web — which agent writes/reads which DB table. TS/JS via Drizzle (db.insert/update/delete, .from(...)); Python via raw SQL (INSERT INTO, UPDATE, FROM, JOIN). camelCase and snake_case names for the same physical table are canonicalised onto one node.
3. Extracts the call web — which agent's functions talk to which other agent.
4. Reads cadence from server/scheduler.ts (intervalMs / CONDITIONAL_AGENTS) and joins it to the tables each agent writes → table freshness.
5. Reads the web's geometry and flags strands that do not look right.
6. Weaves reports into docs/handover/.

The same graph, several webs

One extraction, multiple projections (you don't re-parse to change lens):

• architecture × tables — data-flow web (who writes/reads what).
• architecture × architecture — call web (who talks to whom, weighted).
• architecture × tables × time — table lifecycle + cadence/freshness.

Outputs

• docs/handover/arachne-explorer.html - the primary interface: searchable agents and tables, clickable relationships, ego graphs, glossary, architecture history, change filters, and per-node first-seen/last-changed context.

• docs/handover/arachne-web.html - the architecture health report, static webs, exact run history, metric sparklines, and the latest structural changes.

• docs/handover/arachne-history/*.json - versioned, machine-readable architecture snapshots. arachne-web.json is always the latest snapshot.

• docs/handover/arachne-web.md (+ .json) — the full report, all webs.

• docs/handover/arachne-tables.md — a generated table manifest per agent: the writes: / reads: / cadence: header block you'd otherwise maintain by hand, derived from code so it cannot drift. Paste it atop an agent's entry file (a future --write mode can apply them in place).

Run

npx arachne-web                 # scans your roots, writes the reports (no install)
npx arachne-web workers         # one root
npx arachne-web --focus         # data-flow web shows only high-severity strands
npx arachne-web snapshot --output .arachne/baseline.json
npx arachne-web diff before.json after.json
npx arachne-web --ci --baseline .arachne/baseline.json

From a clone of this repo you can also run npm run arachne or node arachne.mjs.

--focus filters the same live analysis down to high-severity threads — it is not a saved snapshot, so it always reflects the current architecture.

npx arachne-web --agent=states   # ego-web for one agent

--agent=<name> (or npm run arachne:agent -- <name>) prints and writes an HTML viewer for the ego-web of a single agent: that node in the centre with only its own table strands (write ━━red / read ┅┅blue) and its weighted call neighbours (══green, in/out). Name match is fuzzy; a miss suggests near names. These per-agent files are on-demand and git-ignored.

Interactive explorer (the legible view)

The full data-flow web is a hairball at this scale, so report mode also writes docs/handover/arachne-explorer.html — a self-contained, click-to-inspect UI (no build, no server; open the file). Filter the agent/table list on the left; click any node to see its tables (write/read), functions, weighted call neighbours (in/out), bucket and cadence, plus a small ego-web rendered just for that node. Every chip is clickable, so you walk the graph one legible neighbourhood at a time instead of staring at the whole web. A built-in 📖 glossary (shown on open) explains every term — writer, reader, calls out/called by, cadence, the table states, and what the numbers mean.

The output directory is docs/handover/ by default; set outDir in arachne.config.json to write the reports elsewhere.

Each report-mode run sweeps stale Arachne artifacts from outDir (old per-agent ego-webs, leftover offline builds, renamed outputs) so the folder never accumulates cruft. It only ever touches files named arachne-*; pass --keep to preserve them.

Architecture history, diffs, and CI

Every normal report run stores a versioned JSON snapshot in historyDir (default docs/handover/arachne-history). A snapshot contains the schema version, capture time, commit and branch when Git is available, an analysis hash, summary counts, agents, tables, table states, anomalies, functions, and read/write/static-call-site edges. Pass --no-history for an ephemeral report.

Create a snapshot at an explicit path without generating reports or changing history:

arachne snapshot --output .arachne/snapshots/baseline.json

Compare any two v1 snapshots. Human-readable output is the default; --json prints the diff contract and --output saves it:

arachne diff before.json after.json
arachne diff before.json after.json --json --output architecture-diff.json

Diff events include added/removed agents and tables, table state transitions, writer and reader changes, cadence changes, introduced/resolved anomalies, new or removed dependencies, and static coupling changes. A coupling delta is a change in resolved static call sites, not runtime request traffic.

Use the latest stored snapshot as the baseline for a non-mutating CI gate, or pass an explicit baseline:

arachne --ci
arachne --ci --baseline .arachne/snapshots/main.json
arachne --ci --baseline main.json --fail-on new-high,table-regression,new-starved
arachne --ci --baseline main.json --max-call-site-increase 100

The default policy is new-high. Available policies are new-high, new-anomaly, table-regression, new-orphaned, new-starved, and call-site-increase. Configure defaults with ciFailOn and ciMaxCallSiteIncrease. CI exits 1 for policy regressions and 2 when a baseline or option is invalid; it does not write reports or history.

The Explorer's history view exposes exact per-run counts plus searchable change events. Filter for terms such as orphaned, a table or agent name, CALL_EDGE_ADDED, or a specific run. Agent and table pages also show when the node was first seen, last changed, and its recent structural changes.

Offline / air-gapped (--offline)

The viewers normally load Mermaid from a CDN (tiny files, needs internet to draw diagrams). For a fully self-contained copy — air-gapped demo, no network — run:

npm run arachne -- --offline                 # or: node arachne.mjs --offline

This writes additive, git-ignored arachne-web.offline.html and arachne-explorer.offline.html with Mermaid inlined (≈3 MB each), and never touches the small committed CDN versions. The next normal run sweeps them away. Mermaid is resolved from a local source first — --mermaid=<path>, the mermaidJs config key, node_modules/mermaid/dist/mermaid.min.js, then a tmp cache — and only fetched from the CDN as a last resort. If none are available it warns and falls back to CDN so output is never broken.

Reading the call web

Each node is an agent group, not a file — every file under e.g. server/routes/* folds into one server/routes node (that's why nodes look "merged"). The number on an arrow A →|n| B is n resolved cross-agent call sites: places in A that call a function whose definition is owned by B. Resolution is by function name (fnOwner), so it measures relative coupling, not exact dispatch — same-named functions in two agents attribute to whichever was defined first, and built-ins/self-calls are dropped. The full web shows the top 25 edges by weight.

How cadence is determined

Cadence comes only from the scheduler file (scheduleFile, default server/scheduler.ts) — Arachne parses every { name, intervalMs } / cadence: entry and fuzzy-joins it to an agent by folder name. It does not read a schedule out of the agent's own source. So "cadence: —" means not registered in the scheduler (a candidate orphan-from-scheduling), even if the file has an internal timer.

Use it on any repo (standalone)

Arachne is a self-contained, zero-dependency package (one arachne.mjs). Point it at any codebase by dropping an arachne.config.json at the repo root — see arachne.config.example.json for every knob (roots, bucket layout, ORM/SQL patterns, the retired-DB term, the cadence source). Anything you omit falls back to a default.

npx arachne-web                 # run without installing
npm i -g arachne-web            # or install once, then run `arachne-web` anywhere

(npm i arachne-web only installs it; the command you then run is arachne-web.)

Stamp table headers into the agent files

A per-file header listing the tables that file writes/reads (plus the agent's cadence), kept in sync with the code so it can't drift. Scoped to workers/ agent buckets (agents/, scrapers/, crawlers/, housekeeping/), and only files that actually touch a table.

npm run arachne -- --check      # CI gate: exits 1 if any header is missing/stale
npm run arachne -- --write      # stamp / update the headers in place (idempotent)

The block is delimited by // ┌─ arachne … / // └ …; re-running replaces it.

Anomaly types

| Type | Severity | Meaning | |---|---|---| | MULTI_BUCKET_WRITER | 🔴 high | A table written across bucket lines (e.g. workers/ and legacy server/) — migration drift. | | SEVERED_ANCHOR | 🔴 high | A strand reaching a retired database (heliumdb). | | VERIFIER_WRITE | 🔴 high | An audit-only agent mutating non-quarantine data. | | MULTI_WRITER | 🟠 warn | A table written by more than one agent — no single owner. | | ORPHANED_TABLE | 🟠 warn | A table written but never read — dead data. | | STARVED_TABLE | 🟠 warn | A table read but never written — always empty/stale. | | STUB_FUNCTION | 🟠 warn | A trivial / gutted body — the rogue-commit signature. | | ORPHAN_FUNCTION | ⚪ candidate | Defined, no call seen. Low confidence — regex call-tracking is approximate, so orphans are reported separately and never counted as anomalies. |

Known limitations

• Table extraction matches Drizzle (TS/JS) and raw uppercase SQL (Python). ORM calls that don't hit those shapes — e.g. SQLAlchemy session.add(...), lowercase inline SQL — are missed.
• --write header stamping is JS/TS only (.py files use # comments and are skipped for now).
• Orphan detection is regex-based and cannot see dynamic dispatch or scheduler registration, so it is advisory only.
• Bucket/agent identity is derived from path conventions (configurable via arachne.config.json).