paradigmap

A computational method for mapping paradigmatic structure in academic fields.

Given a set of journals, paradigmap fetches their article metadata and citation graph from OpenAlex and CrossRef, embeds abstracts, clusters articles into candidate paradigm groups with seeded k-means, and applies a chi-squared insularity test to measure whether citations cross cluster boundaries more or less often than a null model predicts. Additional passes build a hierarchical stability tree with citation-modularity node signals, project non-core works into the tree, identify bridge articles, run a cross-paradigm co-authorship counterfactual, fit segmented-regression interrupted time-series models, and sub-cluster the external (non-core) literature.

The framework is corpus-agnostic: journals, time windows, cutpoints, and analysis inputs all come from a TOML config. It is published as one package, @easier-idx/paradigmap, with a paradigmap CLI and per-subpath library exports.

The study this framework was extracted from — the reading-paradigm divide (paper, verified results, companion site, Rust accelerators, Python fitters) — lives in the companion repository reading-paradigm-divide.

Runtime: Bun ≥ 1.2 only. Commands use Bun.spawn / import.meta.main, and the published JS keeps extensionless relative imports that Node's ESM loader will not resolve.

Install

bun add @easier-idx/paradigmap

Prerequisites:

• PostgreSQL ≥ 14 with pgvector and pg_trgm
• An embedding provider — an embed-stack host (EMBED_STACK_URL), plus optionally OPENAI_API_KEY for a secondary consensus model
• ANTHROPIC_API_KEY for the two LLM passes (cluster descriptions, bridge classification) — both are skipped cleanly when unset

Quick start

cp "$(bunx paradigmap fixtures-path)/../config.example.toml" config.toml   # or write your own
createdb mystudy && export DATABASE_URL=postgres://localhost/mystudy

bunx paradigmap migrate          # apply db/migrations
bunx paradigmap run              # full pipeline, idempotent — re-run anytime

Every command reads --config <path> (default ./config.toml) and DATABASE_URL from the environment. All outputs (results/…) and relative config paths resolve against your working directory, never the installed package.

Smoke-test the install against the packaged 10-article fixture corpus:

createdb fixture && export DATABASE_URL=postgres://localhost/fixture
bunx paradigmap migrate
bunx paradigmap seed-fixtures
PARADIGMAP_PIPELINE_SKIP=step-01-corpus,step-02-embed,step-05-describe,step-09b-classify-bridges,step-10b-bridge-regression \
  bunx paradigmap run --config "$(bunx paradigmap fixtures-path)/seed.toml"

CLI

bunx paradigmap --help prints the full list. Highlights:

| Command | What it does | |---|---| | run-pipeline (alias run) | every step in order; opens an umbrella pipeline_runs row stamped with git sha, config sha, and framework_version | | step-01-corpus … step-13-viz (aliases step-01…step-13) | any single step; same flags as before, plus --config, and --force where re-runs need it | | migrate | apply the packaged db/migrations to DATABASE_URL | | reset-derived | truncate everything downstream of clustering; keeps corpus + embeddings | | seed-fixtures | load the packaged fixture corpus | | results-summary / results-export | run every read-only export (TSV to stdout/--out-dir, or CSV+MD+XLSX flavors) | | export-* | individual exports (insularity, hierarchy, bridges, time series, …) | | verify-node-signals | reproduce the modularity / k-selection figures from the live DB | | verify-secondary-reproduction --dir <dir> | byte-diff persisted secondary runs against a sidecar partition | | coverage-check, embed-coverage, embed-smoke, … | data-quality and throughput probes | | fixtures-path | print the packaged fixtures directory |

PARADIGMAP_PIPELINE_SKIP is a comma-separated list of step names to skip (step-02-embed; legacy .ts-suffixed names still accepted) — used by CI to bypass network-bound steps when fixtures pre-load the DB.

Configuration

One TOML file drives a study; see config.example.toml for the annotated reference. Sections:

| Section | Required | Purpose | |---|---|---| | [pipeline] | yes | study name + the single seed propagated through every stochastic step | | [corpus] + [[corpus.journals]] | yes | year range and the journal list (id, name, issn, optional issn_print/publisher, free-text role — the framework never branches on role values) | | [corpus.openalex], [corpus.crossref], [corpus.fuzzy] | yes | API rate limits, CrossRef polite-pool mailto (or CROSSREF_MAILTO env), fuzzy-matching thresholds | | [embedding] (+ optional [embedding.secondary]) | yes | primary provider settings; optional second model for cross-model consensus annotation | | [clustering], [projection], [hierarchy], [description], [bridges], [venue_divergence], [regression] | yes | per-pass parameters; clustering.embedding_provider tags every cluster_runs row | | [timeseries] | optional | ITS cutpoint + sensitivity cutpoints. No implicit default — omit the block and steps 08/08b error clearly | | [backends] | optional | bin_dir for prebuilt accelerator binaries (see contract below) | | [analysis] | optional | dir + python_bin for study-side Python sidecars (see contract below) | | [artifacts] | optional | glossary_path feeding the step-12 glossary artifact; absent ⇒ warn and skip |

Contracts

Accelerator binaries ([backends] bin_dir)

The compute-heavy passes (corpus/external/projected k-means, projection, fuzzy reference matching) dispatch through a hybrid runner that prefers prebuilt native binaries and falls back to the pure-TS implementation. The framework knows a flat directory of binaries, never a build system — it never invokes cargo.

• Resolution order: per-step --bin-dir flag → config.backends.bin_dir → PARADIGMAP_BIN_DIR env → unset ⇒ TS backend.
• Expected binary names (flat in that directory): cluster_corpus, cluster_per_journal, cluster_external, cluster_external_projected, project, fuzzy.
• --backend auto (default) probes each binary and falls back to TS on miss; --backend rust with no resolvable bin dir is a hard error, not a silent downgrade; --backend ts never probes.
• The resolved backend is recorded in pipeline_runs.input for every dispatch.

A reference Rust implementation (BLAS-accelerated, bit-identical contract with the TS path at fixed seed) lives in the companion study repository.

Analysis sidecars ([analysis])

The ITS (step-08b) and bridge-regression (step-10b) passes can defer fitting to Python sidecars (statsmodels OLS+HAC / NegBin) when a study provides them:

• Resolution: PARADIGMAP_ANALYSIS_DIR env → config.analysis.dir; interpreter from PARADIGMAP_PYTHON → config.analysis.python_bin → python3 on PATH.
• Expected files in the analysis dir: its.py, bridge_regression.py, bridge-cross-paradigm.py.
• No [analysis] block ⇒ sidecars unresolved ⇒ auto uses the TS fitters (Fisher-scoring NegBin, OLS). An explicit --backend python without a resolvable sidecar is a hard error.
• Regression model 5c (RCS interaction) is sidecar-exclusive and is skipped with a warning on the TS path.
• The resolved backend and sidecar paths are recorded in pipeline_runs.input.

Reproducibility

• One seed per run (pipeline.seed), propagated through k-means++ initialization, silhouette sampling, and permutation nulls. Bit-exact reproducibility at a fixed seed and backend is a hard requirement; TS and native backends are validated against each other in the study repo's parity CI.
• Idempotent and resumable everywhere: upserts by OpenAlex id, UNIQUE (scope, scope_key, k, seed, embedding_provider) on cluster_runs, CAS updates on assignments. run-pipeline can be re-invoked at any point.
• Provenance: every pipeline run stamps {git_sha (caller's repo), config_sha, framework_version, args} into pipeline_runs.input; step-12 writes the same into results/artifacts/manifest.json. Backend choices (rust/ts/python) are recorded per dispatch.
• contentHash dedup on abstracts and a shouldEmbed gate keep embedding calls (the only externally costly step) minimal and idempotent.

Library use

Subpath exports mirror the internal package layout:

import { loadConfig, getPgConnection } from "@easier-idx/paradigmap/config";
import { findExistingRun, getSelectedPartition } from "@easier-idx/paradigmap/cluster-pass";
import { computeFlowMatrix } from "@easier-idx/paradigmap/insularity";

Available subpaths: /config, /corpus, /embed-pass, /cluster-pass, /describe-pass, /hierarchy, /insularity, /project-pass, /timeseries, /bridges, /viz-pass. SQL is the API — raw queries over StoreOps; there is no ORM and none should be added.

Development

git clone <easier> && cd easier && bun install && bun run --filter '*' build
bun run check        # tsc + eslint + build
bun run build        # emit dist/ (what npm ships)

Layout: packages/<name>/src (library code, one domain per package), cli/commands (every runnable step/export/utility; each exports run(argv)), cli/paradigmap.ts (router), db/migrations, fixtures (10-article seed corpus + expected export baselines used by CI).

CI (the paradigmap workflow in this repo) runs the fixture pipeline end-to-end on TS backends and diffs every export against fixtures/expected/, then builds and bun publish --dry-runs, and installs the packed tarball in a bare project to drive the same pipeline through the installed package.

Citation

@misc{paradigmap-2026,
  author       = {Fugal, Russ T.},
  title        = {{paradigmap}: a computational method for mapping paradigmatic structure in academic fields},
  year         = 2026,
  publisher    = {GitHub},
  howpublished = {\url{https://www.npmjs.com/package/@easier-idx/paradigmap}}
}

License

MIT