pi-llm-wiki

Inspired by Andrej Karpathy’s “LLM Wiki” gist: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
This package is a Pi-native implementation of that idea.

Build a persistent, LLM-maintained markdown wiki inside pi with immutable source capture, interlinked knowledge pages, generated navigation metadata, and a bundled wiki-maintainer skill.

pi-llm-wiki implements the “LLM wiki” pattern as a Pi-native package:

• a Pi extension for deterministic operations, guardrails, and generated metadata
• a bundled llm-wiki skill that teaches the model how to maintain the wiki
• a markdown vault structure that accumulates knowledge over time instead of re-deriving it from raw files on every query

Why this package exists

Most file-based LLM workflows behave like one-shot RAG: the model searches raw documents every time you ask a question. That works, but the synthesis is ephemeral.

pi-llm-wiki creates a middle layer:

• raw source packets preserve source-of-truth inputs
• source pages summarize what each source says
• canonical wiki pages track what the wiki currently believes
• generated metadata keeps the whole vault searchable and navigable

The result is a wiki that compounds as you capture sources, ask questions, and file durable analyses.

Features

• Wiki bootstrap — initialize a new vault with config, templates, schema, and metadata files
• Immutable source capture — capture URLs, local files, PDFs, or pasted text into raw/ packets
• Source-page boundary — every source becomes a source page before it influences canonical knowledge
• Canonical page management — safely resolve or create concept, entity, synthesis, and analysis pages
• Generated metadata — rebuilds meta/registry.json, meta/backlinks.json, meta/index.md, and meta/log.md
• Mechanical linting — broken links, orphan pages, duplicate aliases/titles, frontmatter issues, coverage gaps, stale captures
• Operational guardrails — blocks direct edits to raw sources and generated metadata files
• Bundled skill — teaches the model how to capture, integrate, query, and audit the wiki
• Obsidian-friendly links — folder-qualified wikilinks plus stable source-ID citations

Architecture

The vault has four logical layers:

1. Raw capture — immutable source packets in raw/
2. Wiki pages — source pages and canonical pages in wiki/
3. Meta — generated registry, backlinks, index, event log, and lint report in meta/
4. Schema — human/model operating rules in WIKI_SCHEMA.md and .wiki/config.json

Ownership model

| Path | Owner | Rule | |------|-------|------| | raw/** | extension tools | immutable after capture | | wiki/** | model + user | editable knowledge pages | | meta/registry.json | extension | generated | | meta/backlinks.json | extension | generated | | meta/index.md | extension | generated | | meta/events.jsonl | extension/tool | append-only | | meta/log.md | extension | generated from events | | meta/lint-report.md | extension | generated | | WIKI_SCHEMA.md | human + explicit request | operating manual |

Install

From npm:

pi install npm:pi-llm-wiki

From GitHub:

pi install https://github.com/Kausik-A/pi-llm-wiki

Try it without installing:

pi -e https://github.com/Kausik-A/pi-llm-wiki

Quick start

1) Create a new wiki repo or folder

mkdir my-wiki
cd my-wiki
pi

2) Bootstrap the vault

Ask pi:

Initialize an llm wiki here for AI research.

That should call wiki_bootstrap and create:

raw/
wiki/
meta/
.wiki/
WIKI_SCHEMA.md

3) Capture a source

Examples:

Capture this article into the wiki: https://example.com/some-article

Capture this PDF into the wiki: ./papers/context-windows.pdf

Capture these notes into the wiki: ...pasted text...

4) Integrate the source

A good integration flow is:

1. capture the source
2. read wiki/sources/SRC-*.md
3. update that source page
4. search for impacted canonical pages with wiki_search
5. create missing pages with wiki_ensure_page
6. update concept/entity/synthesis pages with citations
7. mark the integration with wiki_log_event kind=integrate

5) Query the wiki

Based on the wiki, what are the main tradeoffs between long-context models and RAG?

By default, the bundled skill treats query mode as read-only.

If you want a durable answer filed back into the vault:

Answer the question and file the result as an analysis page.

Vault layout

my-wiki/
├─ raw/
│  └─ sources/
│     └─ SRC-2026-04-04-001/
│        ├─ manifest.json
│        ├─ original/
│        ├─ extracted.md
│        └─ attachments/
├─ wiki/
│  ├─ sources/
│  ├─ concepts/
│  ├─ entities/
│  ├─ syntheses/
│  └─ analyses/
├─ meta/
│  ├─ registry.json
│  ├─ backlinks.json
│  ├─ index.md
│  ├─ events.jsonl
│  ├─ log.md
│  └─ lint-report.md
├─ .wiki/
│  ├─ config.json
│  └─ templates/
└─ WIKI_SCHEMA.md

Linking and citation style

Internal navigation

Use folder-qualified wikilinks:

[[concepts/retrieval-augmented-generation]]
[[entities/openai|OpenAI]]
[[syntheses/long-context-vs-rag]]

Factual citations

Use stable source page ID links:

[[sources/SRC-2026-04-04-001|SRC-2026-04-04-001]]

This keeps provenance stable even if titles or page summaries change.

Tools

| Tool | Description | |------|-------------| | wiki_bootstrap | Initialize the vault structure, config, templates, schema, and metadata files | | wiki_capture_source | Capture a URL, file, or pasted text into an immutable source packet and create a source page | | wiki_search | Search the generated wiki registry | | wiki_ensure_page | Resolve or safely create canonical concept/entity/synthesis/analysis pages | | wiki_lint | Run deterministic health checks over the wiki | | wiki_status | Show counts, source states, and recent activity | | wiki_log_event | Append structured events and regenerate meta/log.md | | wiki_rebuild_meta | Force a full metadata rebuild |

Commands

| Command | Description | |---------|-------------| | /wiki-status | Show a concise operational summary | | /wiki-lint [mode] | Run mechanical lint (all, links, orphans, frontmatter, duplicates, coverage, staleness) | | /wiki-rebuild | Force a full metadata rebuild |

Guardrails

The extension blocks direct edits to:

• raw/**
• meta/registry.json
• meta/backlinks.json
• meta/events.jsonl
• meta/index.md
• meta/log.md
• meta/lint-report.md

If the model directly edits wiki/** using Pi’s built-in write or edit tools, pi-llm-wiki automatically rebuilds generated metadata at the end of the agent turn.

Source packet format

Each captured source is stored as a packet:

raw/sources/SRC-YYYY-MM-DD-NNN/
├─ manifest.json
├─ original/
├─ extracted.md
└─ attachments/

This lets you preserve:

• the original artifact
• normalized extracted text for reading
• capture metadata
• future attachment downloads

Page model

Source pages

wiki/sources/SRC-*.md

These answer: what does this specific source say?

Canonical pages

• wiki/concepts/ — concepts and recurring ideas
• wiki/entities/ — people, orgs, products, papers, systems
• wiki/syntheses/ — cross-source theses and tensions
• wiki/analyses/ — durable filed answers from queries

These answer: what does the wiki currently believe?

Skill behavior

The bundled llm-wiki skill teaches Pi to:

• never edit raw sources directly
• treat generated metadata as machine-owned
• capture first, integrate second
• search before creating new canonical pages
• cite facts using source-page IDs
• keep query mode read-only by default
• use Tensions / caveats and Open questions when evidence is mixed

Versioning and releases

This package uses Semantic Versioning and includes a release/tag flow built for repeatable publishes.

Release flow

1. Add notes under ## [Unreleased] in CHANGELOG.md
2. Run one of:

npm run release:patch
npm run release:minor
npm run release:major

This will:

• verify the git working tree is clean
• verify you are on main
• run npm run check
• bump the package version
• move Unreleased notes into a dated version section in CHANGELOG.md
• create a release commit
• create a matching git tag like v0.1.1

3. Push the release commit and tag:

npm run release:push

4. GitHub Actions publishes the tagged version to npm and creates a GitHub Release.

GitHub Actions

This repo includes:

• CI on push and pull request: runs npm ci, npm run check, and npm pack --dry-run
• Release on v* tags: runs checks, publishes to npm, and creates a GitHub release with generated notes

For a fuller walkthrough, see RELEASING.md.

First-time npm publishing checklist

Before the first automated release, do this once:

1. Ensure the npm package name is available:

npm view pi-llm-wiki version

2. Log in locally if you want to do a manual first publish:

npm login

3. Create an npm automation token in npm:

   • npmjs.com → Account Settings → Access Tokens
   • Create a token with publish permission for pi-llm-wiki

4. Add the token as a GitHub repository secret:

gh secret set NPM_TOKEN --repo Kausik-A/pi-llm-wiki

5. Optionally do the first release manually:

npm run check
npm publish --access public

6. After that, use the release/tag flow for future versions.

Required repository secret

To enable npm publishing from GitHub Actions, add this repository secret:

• NPM_TOKEN — an npm access token with publish permissions for pi-llm-wiki

Manual fallback publish

If you ever need to publish manually:

npm run check
npm publish --access public

Then users can update with:

pi update

Local development

Install locally for testing:

pi install ./pi-llm-wiki

Load only the extension for one-off testing:

pi -e ./pi-llm-wiki/extensions/llm-wiki/index.ts

Sanity-check the package:

cd pi-llm-wiki
npm run check

Notes

• For PDFs and some binary formats, the extension tries uvx --from 'markitdown[pdf]' markitdown ... when available.
• If markitdown is unavailable, capture falls back to simpler text or placeholder extraction.
• v1 intentionally avoids embeddings and vector databases; the wiki itself is the main retrieval layer.

License

MIT