dataform-doc-generator

Interactive, credential-free documentation for Dataform projects. Compiles your project with dataform compile --json (offline) and renders a static site with a dependency graph, column-aware search, and assertion linkage.

Published to npm as dataform-doc-generator. If you fork it, change name+bin in package.json and the program name in src/cli/index.ts.

Improvements over the original dataform-docs

• Column-aware search — the search box and list filter match column names and column descriptions, not just model name/description/tags (src/utils/search.ts).
• Assertion linkage — each model's detail panel lists the assertions that test it, and assertions link back to what they assert against (src/generator/manifest-builder.ts, ModelDetails.tsx).
• Directional lineage isolation — selecting a model shows only its real lineage (transitive upstream dependencies + downstream dependents), not the whole connected blob. Siblings that merely share an ancestor are excluded (lineageSet in DependencyGraph.tsx).
• Zoom-to-fit on select — selecting a model animates a zoom onto its lineage so it's easy to read; the "Show all" view fits the whole graph without over-zooming.
• Collapsible model list — the left list groups by type with collapsible sections and count badges. Assertions, declarations and operations start collapsed; they auto-expand while a search or filter is active (ModelList.tsx).

Layout

src/                 CLI + parser (TypeScript → dist/)
  cli/               index (commander), generate, serve
  parser/            dataform-compiler (compile --json), dependency-graph
  generator/         manifest-builder (+ assertion reverse-map)
  types/
site-generator/      React + Vite SPA (built once, copied into src/site at build time)
scripts/copy-site.js copies the built SPA into src/site so it ships in the package

Develop

npm install
npm run build:cli          # compile the CLI to dist/
cd site-generator && npm install && npm run dev   # hot-reload the SPA against sample JSON

Build & publish

npm install
npm run build              # tsc + build SPA + copy SPA into src/site
npm publish --access public

Use (against any Dataform project)

The generator shells out to a dataform binary, so the Dataform CLI must be on your PATH:

npm install -g @dataform/cli@^3

Then, from your project (before publishing, run the local build with node …/dist/cli/index.js):

cd /path/to/dataform-project
dataform compile --json | head    # smoke test: should print JSON, not an error
npx dataform-doc-generator generate --project . --output ./site
npx dataform-doc-generator serve --dir ./site --open

Dataform 3.x note

3.x projects use workflow_settings.yaml and have no package.json — that's expected, don't run npm install in them. If a package-lock.json is present, dataform compile fails with '…/package-lock.json' unexpected; remove it and try again; delete it and recompile.

For GitHub Pages, generate with the repo subpath:

npx dataform-doc-generator generate -o ./site -b "/your-repo-name/"

Notes

• Compile-only: column types are not available offline, so they show as unknown. Descriptions, tags, partition/cluster, dependencies and assertions are all real.
• CI needs no secrets. See the companion docs.yml workflow.