LibScope

LibScope is a local knowledge base that makes your documentation searchable by AI assistants. Point it at markdown files, URLs, or connect it to Obsidian/Notion/Confluence/Slack, and it chunks, embeds, and indexes everything into a local SQLite database. Your AI tools query it through MCP (Model Context Protocol) or a REST API.

Everything runs on your machine. No cloud services required for basic usage — just npm install and go.

Getting Started

npm install -g libscope

# Set up the database
libscope init

# Index some docs
libscope add ./docs/getting-started.md --library my-lib
libscope add https://docs.example.com/guide
libscope import ./docs/ --library my-lib --extensions .md,.mdx

# Search
libscope search "how to authenticate"

# Start the MCP server so your AI assistant can query it
libscope serve

On first run with the default embedding provider, LibScope downloads the all-MiniLM-L6-v2 model (~80 MB). Subsequent runs use the cached model.

Document Format Support

LibScope supports Markdown (.md, .mdx) and plain text natively. Additional formats are available via optional dependencies:

| Format | Extension | Optional Dependency | Node.js Requirement | | ------------ | --------- | ------------------- | ------------------- | | PDF | .pdf | pdf-parse (v2) | ≥ 20.16 or ≥ 22.3 | | Word | .docx | mammoth | Any | | EPUB | .epub | epub2 | Any | | PowerPoint | .pptx | pizzip | Any | | CSV | .csv | Built-in | Any |

The pdf-parse, mammoth, epub2, and pizzip packages are listed as optionalDependencies and install automatically when the Node.js version is compatible. Note: binary .ppt files are not supported — only .pptx.

Using with AI Assistants

LibScope exposes an MCP server over stdio. Point your MCP-compatible client at it:

Cursor — add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "libscope": {
      "command": "npx",
      "args": ["-y", "libscope", "serve"]
    }
  }
}

Claude Code:

claude mcp add --scope user libscope -- npx -y libscope serve

Once connected, your assistant can search docs, submit new documents, rate content quality, and ask RAG-powered questions against your knowledge base.

| Tool | What it does | | ---------------------- | --------------------------------------------------------- | | search-docs | Semantic search with topic/library/version/rating filters | | ask-question | RAG question-answering with source citations | | get-document | Retrieve a document by ID | | list-documents | List docs with filters | | list-topics | Browse the topic hierarchy | | submit-document | Index new content (raw text or a URL to fetch) | | update-document | Update a document's title, content, or metadata | | delete-document | Remove a document | | rate-document | Rate a doc 1–5 with optional feedback and corrections | | suggest-tags | Auto-suggest tags based on content analysis | | save-search | Save a named search query with filters | | list-saved-searches | List all saved searches | | run-saved-search | Execute a saved search by name or ID | | delete-saved-search | Delete a saved search | | link-documents | Create a cross-reference between two documents | | get-document-links | List all incoming and outgoing links for a document | | delete-link | Remove a cross-reference link | | reindex-documents | Re-embed chunks (useful after switching providers) | | health-check | DB status, doc/chunk counts | | sync-obsidian-vault | Sync an Obsidian vault | | sync-onenote | Sync OneNote notebooks via Microsoft Graph | | sync-notion | Sync Notion pages and databases | | sync-confluence | Sync Confluence spaces | | sync-slack | Sync Slack channels and threads | | install-pack | Install a knowledge pack | | list-packs | List installed or registry packs |

Connectors

LibScope can pull documentation from several platforms. Each connector handles incremental syncing so re-runs only process what changed.

# Obsidian — parses wikilinks, frontmatter, embeds, tags
libscope connect obsidian /path/to/vault
libscope connect obsidian /path/to/vault --sync   # incremental re-sync

# Notion
libscope connect notion --token secret_abc123

# Confluence
libscope connect confluence \
  --url https://acme.atlassian.net \
  --email [email protected] \
  --token $CONFLUENCE_TOKEN

# Slack — index channel messages and threads
libscope connect slack --token xoxb-... --channels general,engineering

# OneNote — device code auth via Microsoft Graph
export ONENOTE_CLIENT_ID=your-client-id
libscope connect onenote

# GitHub / GitLab repos
libscope add-repo https://github.com/org/repo --branch main --path docs/

# Remove a connector's data
libscope disconnect obsidian /path/to/vault

Obsidian: --topic-mapping frontmatter, --exclude "templates/*" "daily/*", --sync

Notion: --exclude page-id-1 db-id-2, --sync

Confluence: --spaces ENG,DEVOPS, --exclude-spaces ARCHIVE

Slack: --thread-mode aggregate|separate, --sync

OneNote: --notebook "Work Notes", --sync

GitHub/GitLab: --token, --branch, --path, --extensions .md,.mdx,.rst

Search and RAG

# Semantic search
libscope search "authentication best practices"
libscope search "API rate limiting" --library my-lib --topic security --limit 10

# Ask questions (needs an LLM provider configured — see Configuration below)
libscope ask "How do I configure OAuth2?" --library my-lib

# Interactive REPL for iterative searching
libscope repl

Search uses sqlite-vec for vector similarity when available, with FTS5 full-text search as a fallback. Results automatically get a 1.5× title boost when the document title matches query words. You can also pass --diversity 0.5 (0–1) for MMR-based diversity reranking.

Programmatic SDK

LibScope exports two embeddable APIs:

LibScope — full SDK with all features (connectors, topics, packs, etc.):

import { LibScope } from "libscope";

const scope = LibScope.create();
await scope.index({ title: "My Doc", content: "..." });
const results = await scope.search("query");
scope.close();

LibScopeLite — lightweight embeddable class for external applications. No CLI, no MCP server, no connectors. Designed for embedding semantic search directly into other tools (MCP servers, VS Code extensions, CI scripts):

import { LibScopeLite } from "libscope/lite";

const lite = new LibScopeLite({ dbPath: ":memory:" });

// Index documents (or code files via tree-sitter chunking)
await lite.indexBatch(docs, { concurrency: 4 });

// Hybrid vector + FTS5 search
const results = await lite.search("how to authenticate");

// Get RAG context for injection into an external LLM prompt
const context = await lite.getContext("How does auth work?");

lite.close();

Tree-sitter powered code indexing splits TypeScript, JavaScript, and Python files at function/class boundaries:

import { TreeSitterChunker } from "libscope/lite";

const chunker = new TreeSitterChunker();
const chunks = await chunker.chunk(sourceCode, "typescript");
// Each chunk is a complete function or class with 1-based line numbers

See the LibScope Lite guide and API reference for the full documentation.

Organizing Content

Topics give your docs a hierarchy. Tags give them flexible labels. Workspaces give you isolated databases.

# Topics
libscope topics create "backend"
libscope topics create "auth" --parent backend --description "Auth & identity"

# Tags
libscope tag add <doc-id> typescript,api,v2

# Workspaces — separate knowledge bases entirely
libscope workspace create my-project
libscope workspace use my-project
libscope --workspace my-project search "deploy steps"

Documents also keep version history, so you can roll back if a re-index goes wrong:

libscope docs history <doc-id>
libscope docs rollback <doc-id> 3

REST API

For programmatic access outside of MCP:

libscope serve --api --port 3378

OpenAPI 3.0 spec at GET /openapi.json. Key endpoints:

| Method | Endpoint | Description | | --------------- | --------------------------------- | ---------------------------------- | | GET | /api/v1/search?q=... | Semantic search | | POST | /api/v1/batch-search | Batch search (up to 20 queries) | | POST | /api/v1/ask | RAG question-answering | | GET/POST | /api/v1/documents | List or create documents | | GET/PATCH/DELETE | /api/v1/documents/:id | Get, update, or delete a document | | POST | /api/v1/documents/url | Index from a URL | | POST | /api/v1/documents/:id/tags | Add tags | | GET | /api/v1/documents/:id/suggest-tags | Auto-suggest tags | | GET/POST | /api/v1/documents/:id/links | List or create cross-references | | DELETE | /api/v1/links/:id | Delete a cross-reference | | GET/POST | /api/v1/topics | List or create topics | | GET | /api/v1/tags | List tags | | GET/POST | /api/v1/searches | List or create saved searches | | POST | /api/v1/searches/:id/run | Run a saved search | | DELETE | /api/v1/searches/:id | Delete a saved search | | POST | /api/v1/bulk/delete | Bulk delete documents | | POST | /api/v1/bulk/retag | Bulk add/remove tags | | POST | /api/v1/bulk/move | Bulk move to a topic | | GET/POST | /api/v1/webhooks | List or create webhooks | | DELETE | /api/v1/webhooks/:id | Delete a webhook | | POST | /api/v1/webhooks/:id/test | Send a test ping to a webhook | | GET | /api/v1/analytics/searches | Search analytics and knowledge gaps| | GET | /api/v1/connectors/status | Connector sync status and history | | GET | /api/v1/stats | Usage statistics | | GET | /api/v1/health | Health check |

Configuration

LibScope reads config from (highest priority first): environment variables → .libscope.json in your project → ~/.libscope/config.json → built-in defaults.

Embedding Providers

| Provider | Default? | Requirements | Model | | -------- | -------- | ----------------------------------------- | ---------------------- | | local | Yes | None (~80 MB model download on first run) | all-MiniLM-L6-v2 | | ollama | | Ollama running locally | nomic-embed-text | | openai | | LIBSCOPE_OPENAI_API_KEY | text-embedding-3-small |

libscope config set embedding.provider ollama
# or
export LIBSCOPE_EMBEDDING_PROVIDER=openai
export LIBSCOPE_OPENAI_API_KEY=sk-...

LLM for RAG

The ask command and ask-question MCP tool need an LLM. Configure one with:

export LIBSCOPE_LLM_PROVIDER=openai   # or ollama, anthropic
export LIBSCOPE_LLM_MODEL=gpt-4o-mini # optional

| Variable | Description | Default | | ---------------------------------- | ---------------------------------------- | ------------------------ | | LIBSCOPE_EMBEDDING_PROVIDER | local, ollama, or openai | local | | LIBSCOPE_OPENAI_API_KEY | OpenAI API key | — | | LIBSCOPE_OLLAMA_URL | Ollama server URL | http://localhost:11434 | | LIBSCOPE_LLM_PROVIDER | LLM for RAG (openai / ollama / anthropic) | — | | LIBSCOPE_LLM_MODEL | LLM model override | — | | LIBSCOPE_ANTHROPIC_API_KEY | Anthropic API key (for Claude models) | — | | LIBSCOPE_ALLOW_PRIVATE_URLS | Allow fetching from private/internal IPs | false | | LIBSCOPE_ALLOW_SELF_SIGNED_CERTS | Accept self-signed TLS certificates | false | | ONENOTE_CLIENT_ID | Microsoft app registration client ID | — | | ONENOTE_TENANT_ID | Microsoft tenant ID | common | | NOTION_TOKEN | Notion integration token | — | | CONFLUENCE_URL | Confluence base URL | — | | CONFLUENCE_EMAIL | Confluence user email | — | | CONFLUENCE_TOKEN | Confluence API token | — |

{
  "embedding": {
    "provider": "local",
    "ollamaUrl": "http://localhost:11434",
    "ollamaModel": "nomic-embed-text",
    "openaiModel": "text-embedding-3-small"
  },
  "llm": {
    "provider": "openai",
    "model": "gpt-4o-mini",
    "anthropicApiKey": "sk-ant-..."
  },
  "database": {
    "path": "~/.libscope/libscope.db"
  },
  "logging": {
    "level": "info"
  },
  "indexing": {
    "allowPrivateUrls": false,
    "allowSelfSignedCerts": false
  }
}

Corporate / Internal Networks

If you're indexing docs from internal servers (Confluence, wikis, etc.), you may need:

# Allow fetching from private/internal IP addresses
libscope config set indexing.allowPrivateUrls true

# Accept self-signed or corporate TLS certificates
libscope config set indexing.allowSelfSignedCerts true

Or via environment variables:

export LIBSCOPE_ALLOW_PRIVATE_URLS=true
export LIBSCOPE_ALLOW_SELF_SIGNED_CERTS=true

Webhooks

LibScope can push events to any HTTP endpoint. Useful for triggering CI pipelines, Slack notifications, or custom workflows whenever documents are created or updated.

libscope serve --api  # webhooks require the REST API

# Create a webhook
curl -X POST http://localhost:3378/api/v1/webhooks \
  -H "Content-Type: application/json" \
  -d '{"url": "https://hooks.example.com/libscope", "events": ["document.created", "document.updated"], "secret": "my-hmac-secret"}'

# Send a test ping
curl -X POST http://localhost:3378/api/v1/webhooks/<id>/test

Webhook payloads are signed with HMAC-SHA256 when a secret is set. The signature is in the X-LibScope-Signature header.

Supported events: document.created, document.updated, document.deleted.

Other Tools

LibScope ships with a few more utilities beyond the core index-and-search loop:

# Watch a directory and auto-reindex on changes
libscope watch ./docs/

# Re-embed everything after switching embedding providers
libscope reindex

# Find duplicate documents
libscope dedupe

# Export / import the whole knowledge base
libscope export ./backup.json
libscope import-backup ./backup.json

# Usage analytics
libscope stats                  # overview
libscope stats popular          # most-searched docs
libscope stats stale --days 90  # docs nobody searches for

# Knowledge packs — portable document bundles
libscope pack create --name "react-docs" --topic react
libscope pack install ./react-docs.json

Pack Registries

Share and discover knowledge packs through git-based registries. A registry is a git repo with a defined folder structure managed by libscope.

# Add a registry
libscope registry add https://github.com/org/libscope-registry.git --name official

# Search for packs across all registries
libscope registry search "react"

# Install a pack by name (resolves from registries)
libscope pack install react-docs
libscope pack install [email protected]    # specific version

# Create your own registry
libscope registry create ./my-registry

# Publish a pack file to your registry
libscope registry publish ./my-pack.json -r my-registry --version 1.0.0

# Submit a pack to someone else's registry (creates a feature branch)
libscope registry publish ./my-pack.json -r community --submit

Authentication is delegated to git — SSH keys and HTTPS credential helpers work automatically. Registries cache locally and support offline index lookups. See the Pack Registries guide for full details.

There's also a web dashboard at http://localhost:3377 when you run libscope serve, with search, document browsing, topic navigation, and a knowledge graph visualization at /graph.

Core

| Command | Description | | ----------------------------------- | ----------------------------------- | | libscope init | Initialize the database | | libscope add <fileOrUrl> | Index a file or URL | | libscope import <directory> | Bulk import from a directory | | libscope import-batch <directory> | Parallel batch import | | libscope search <query> | Search | | libscope ask <question> | RAG question-answering | | libscope repl | Interactive search REPL | | libscope serve | Start MCP server (--api for REST) |

Documents

| Command | Description | | --------------------------------------- | ---------------------------- | | libscope docs list | List documents | | libscope docs show <id> | Show a document | | libscope docs update <id> | Update title/content/metadata| | libscope docs delete <id> | Delete a document | | libscope docs history <id> | Version history | | libscope docs rollback <id> <ver> | Roll back to a prior version |

Organization

| Command | Description | | ------------------------------------ | -------------------------------- | | libscope topics list | List topics | | libscope topics create <name> | Create a topic | | libscope tag add <id> <tags...> | Add tags | | libscope tag remove <id> <tag> | Remove a tag | | libscope tag list | List tags | | libscope workspace create <name> | Create workspace | | libscope workspace list | List workspaces | | libscope workspace use <name> | Switch workspace | | libscope workspace delete <name> | Delete workspace |

Saved Searches

| Command | Description | | ------------------------------------ | ----------------------------- | | libscope searches list | List all saved searches | | libscope searches run <name> | Re-run a saved search | | libscope searches delete <name> | Delete a saved search | | libscope search <q> --save <name> | Save a search while running it|

Document Links

| Command | Description | | ------------------------------------- | -------------------------------- | | libscope link <srcId> <tgtId> | Create a cross-reference | | libscope links <docId> | Show all links for a document | | libscope unlink <linkId> | Remove a link | | libscope prereqs <docId> | Show prerequisite reading chain |

Bulk Operations

| Command | Description | | ---------------------------- | ----------------------------------- | | libscope bulk delete | Delete all matching documents | | libscope bulk retag | Add/remove tags on matching docs | | libscope bulk move | Move matching docs to a topic |

Connectors

| Command | Description | | ---------------------------------- | -------------------------- | | libscope connect obsidian <path> | Sync Obsidian vault | | libscope connect onenote | Sync OneNote | | libscope connect notion | Sync Notion | | libscope connect confluence | Sync Confluence | | libscope connect slack | Sync Slack | | libscope add-repo <url> | Index a GitHub/GitLab repo | | libscope disconnect <name> | Remove connector data |

Registries

| Command | Description | | ----------------------------------------------------- | ---------------------------------------- | | libscope registry add <url> [-n <alias>] | Register a git repo as a pack registry | | libscope registry remove <name> | Unregister a registry | | libscope registry list | List configured registries | | libscope registry sync [<name>] | Sync one or all registries | | libscope registry search <query> [-r <name>] | Search registry pack indexes | | libscope registry create <path> | Initialize a new registry repo | | libscope registry publish <file> -r <name> | Publish a pack file to a registry | | libscope registry unpublish <pack> -r <name> | Remove a pack version from a registry |

Utilities

| Command | Description | | --------------------------------- | ---------------------------- | | libscope watch <dir> | Auto-reindex on file changes | | libscope reindex | Re-embed all chunks | | libscope dedupe | Find duplicates | | libscope export <path> | Export to JSON | | libscope import-backup <path> | Import from backup | | libscope stats | Usage overview | | libscope pack install <name> | Install a knowledge pack | | libscope pack create | Create a knowledge pack | | libscope config set <key> <val> | Set config | | libscope config show | Show config |

Global flags: --verbose, --log-level <level>, --workspace <name>

How It Works

LibScope stores everything in a local SQLite database (at ~/.libscope/libscope.db by default):

• Documents are split into chunks by heading boundaries
• Each chunk is embedded into a vector using the configured provider
• Vector search is done via sqlite-vec; FTS5 full-text search is used as a fallback
• The MCP server reads from this same database over stdio
• Connectors fetch content from external platforms and feed it through the same indexing pipeline

The stack: better-sqlite3 + sqlite-vec + @modelcontextprotocol/sdk + @xenova/transformers for local embeddings.

Contributing

See CONTRIBUTING.md. The short version:

git clone https://github.com/RobertLD/libscope.git
cd libscope && npm install
npm run dev        # watch mode
npm test           # run tests
npm run typecheck  # type check
npm run lint       # lint

License

Business Source License 1.1 — see LICENSE for full terms.