@abraca/wiki

Streaming Wikipedia → Abracadabra importer. Fetches Wikipedia (or any MediaWiki) articles and seeds them into a space as a live graph of CRDT docs: every discovered title becomes a shell doc immediately, then bodies stream in one fetch at a time — so the tree skeleton appears in the dashboard before the first body is written.

Extracted from @abraca/cli (where it used to be the abracadabra wiki command) so the heavyweight wtf_wikipedia dependency no longer ships with the core CLI.

Install

pnpm add -g @abraca/wiki   # or: pnpm dlx @abraca/wiki ...

Requires @abraca/dabra (peer). Authenticates with the same Ed25519 key file as the abracadabra CLI (~/.abracadabra/cli.key), so both tools share one account.

Usage

abracadabra-wiki "<Article Title>" user-agent="you ([email protected])" [options]

| Option | Default | Meaning | | :-- | :-- | :-- | | mode=single\|split | split | one doc per article, or split into sections + infobox | | depth=<n> | 1 | follow internal links to depth N | | category-depth=<n> | 1 | recurse into sub-categories | | lang=<code> | en | wiki language | | domain=<host> | — | 3rd-party MediaWiki host (overrides lang) | | parent=<docId> | active space root | parent doc for the new graph | | user-agent=<str> | — | required Api-User-Agent (Wikimedia etiquette) | | rate=<rps> | 3 | max Wikipedia requests per second | | --include-categories | off | expand each article's categories into nested graphs | | --dry-run | off | fetch only the entry article, print its outline, write nothing |

Environment

| Var | Purpose | | :-- | :-- | | ABRA_URL | server URL (required unless --dry-run) | | ABRA_KEY_FILE | Ed25519 key seed (default ~/.abracadabra/cli.key) | | ABRA_NAME / ABRA_COLOR | presence identity | | ABRA_INVITE_CODE | invite code for first-run registration | | ABRA_WIKI_USER_AGENT | default Api-User-Agent (or pass user-agent=) |

Examples

# Preview the structure without touching a server
abracadabra-wiki "Toronto Raptors" user-agent="me ([email protected])" --dry-run

# Import two link-hops deep, split into section docs, with categories
export ABRA_URL=https://your-server.example.com
abracadabra-wiki "Jazz" depth=2 mode=split --include-categories \
  user-agent="me ([email protected])"

Programmatic use

import { runWiki } from "@abraca/wiki";

const summary = await runWiki({
  positional: ["Toronto Raptors"],
  params: { "user-agent": "me ([email protected])" },
  flags: new Set(["dry-run"]),
});

runWiki(args) runs an import for already-parsed args and returns a summary string. (The package is primarily a CLI; abracadabra-wiki is the bin.)

License

MIT