@maschinenlesbar.org/openka-cli
v0.10.0
Published
OpenKA — German parliamentary Kleine Anfragen from 17 incompatible systems in one reproducible, machine-readable schema. Deterministic runtime: no generative model on the line.
Downloads
1,457
Maintainers
Readme
@maschinenlesbar.org/openka-cli
The published package: the library entry point and the two bins.
This is the one package that ships. Everything else in the workspace is private
and is bundled into this tarball at pack time.
It holds almost no code:
src/index.tsre-exports the library surface — the schema and its validators, the store, the deterministic extractors, the PDF reader, the source clients and the reproducibility checks. What is deliberately not exported is the factory.src/bin/ka.tsandsrc/bin/ka-factory.tsare one-line shims, so the tarball has a stable entry point whatever the layout underneath it is.test/pipeline.test.tsis the cross-package integration suite. It belongs here rather than to any one package because it spans them: discover → fetch → extract → normalize → store, and thenka verifyagainst real goldens.
Packing has one wrinkle, and it is worth knowing about
Being a workspace package like every other keeps the repository layout free of
exceptions — and costs one thing. npm bundles bundleDependencies from the
package's own node_modules, and in a workspace the dependencies are symlinked
into the root node_modules instead. Packing from here therefore produces a
tarball with none of them and bins that cannot resolve a single import. The failure
is silent: npm pack succeeds and the tarball is simply 20 kB instead of 300.
tools/prepack.mjs handles it. It copies each workspace dependency's manifest and
built dist into this package's node_modules, walks the dependency graph
transitively — bundling only the direct ones leaves half the graph missing — and
copies in the repository-level documents the tarball carries, LICENSE among them,
because npm cannot reach outside a package directory when it builds one. Everything
it writes is generated and gitignored; the originals stay the single source of truth.
Third-party dependencies are bundled too — today that is commander alone.
prepack copies the runtime closure of every bundled package from the root
node_modules, and refuses to pack when one is missing from bundleDependencies.
Left unbundled, npm install -g created node_modules/commander as an empty
directory, counted it as installed, and every ka command crashed with
ERR_MODULE_NOT_FOUND; a local install hoisted a real copy and hid the bug
(issue #1). test/bundle.test.ts checks the manifest, and CI installs the packed tarball
globally, offline, and runs both bins.
tools/postpack.mjs removes it all again as soon as the tarball exists.
Three more things happen on the way, each because the registry reads the tarball differently from a maintainer reading this directory:
- The README is swapped. npm shows this directory's
README.mdon the package page, and this file is written for whoever maintains the package, not whoever installs it. For the length of the pack the repository README stands in for it, and this one waits aspackage-readme.parked.md. A pack that dies beforepostpackleaves it parked; the nextprepackputs it back first. A relative link in the repository README must therefore name a document the tarball carries (one of prepack'sDOCUMENTS, listed infiles); any other goes by its absolute GitHub URL.test/readme-links.test.tschecks it. - No source maps.
tsconfig.base.jsonemits none, because they would point atsrc/*.ts, which the tarball does not carry. Any an older build left in adist/stay out anyway:filesexcludes this package's own andprepackskips the bundled ones. - Bundled manifests name their licence. A workspace package never ships alone,
so its
package.jsonhas nolicense,authororrepository. A consumer's licence scanner reads every bundled manifest on its own, soprepackstamps those fields onto each copy from this package's manifest.
.npmignore is only a second line of defence behind files. Its patterns are
anchored to this directory, because an unanchored src/ would also match
dist/src/.
Verify a change to any of that the only way that means anything:
npm run pack # from the workspace root
cd /tmp && mkdir t && cd t && npm init -y
npm install /path/to/maschinenlesbar.org-openka-cli-<version>.tgz
./node_modules/.bin/ka sources list
# and globally, which is where an unbundled dependency shows up
npm install -g --offline --prefix /tmp/g /path/to/maschinenlesbar.org-openka-cli-<version>.tgz
/tmp/g/bin/ka --versionDepends on
cli-ka— thekacommand tree and its I/O seamcli-ka-factory— the factory toolingconnector-berlin— the berlin connectorconnector-bund— the bund connectorconnector-niedersachsen— the niedersachsen connectorconnector-nordrhein-westfalen— the nordrhein-westfalen connectorconnector-saarland— the saarland connectorconnector-sachsen— the sachsen connectorconnector-thueringen— the thueringen connectorlib-errors— the shared error hierarchylib-extract— the deterministic tier stacklib-http— the Transport seam and the fetch enginelib-models— the canonical record schema, validators and the parliament tablelib-pardok— theParlamentsspiegel Export 1.0readerlib-parlamentsspiegel— the shared aggregator adapterlib-pdf— the PDF readerlib-perceive— the Perceiver seam (OCR)lib-pipeline— discover → fetch → extract → normalize → storelib-registry— the source registrylib-render— the output rendererslib-repro— canonical JSON, hashing and the extractor version stamplib-search— keyword and semantic searchlib-source— theSourceprotocol and the scraping helperslib-store— the corpus seamlib-verify— re-extract-and-compare, behindka verifycommander— argument parsing — the only third-party runtime dependency in the project
Tests
test/pipeline.test.ts, plus test/readme-links.test.ts and test/bundle.test.ts
for what the tarball carries — run with:
npm test -w @maschinenlesbar.org/openka-cli