@tryformation/querylight-cli

v0.3.6

Published

15 hours ago

Querylight CLI for building and querying local knowledge bases.

0High
0Medium
0Low

Querylight CLI

Querylight CLI is a TypeScript command line application for building and querying local knowledge bases with Querylight TS.

Package: @tryformation/querylight-cli
Binary: qli
Runtime: Node.js 22+

It is designed for local, inspectable workflows:

ingest files, directories, URLs, and websites
normalize content into Markdown-like text
chunk documents for retrieval
build a portable local Querylight index
package a workspace as a zip for deployment
search and generate retrieval context for external agents and tools
serve an OpenSearch-like _search API over one or more local knowledge bases
inspect workspace state, diffs, and change reports

Install

Run without installing globally:

bunx --bun @tryformation/querylight-cli init

For agent and Python automation examples that use bunx and uv, see examples/skills/qli-bunx-uv/SKILL.md.

Install as a dependency:

npm install @tryformation/querylight-cli

Then run:

npx qli --help

If you prefer to avoid a local install, use:

bunx --bun @tryformation/querylight-cli --help

Use bunx --bun for repeated or concurrent bunx calls. bunx respects the CLI shebang by default and otherwise starts qli through node.

Release

Publish releases from semantic version tags such as 0.1.1.

The GitHub Actions publish workflow publishes @tryformation/querylight-cli to the public npm registry.

The publish workflow builds the package and verifies that the built CLI JSON envelope reports the same version as package.json before it publishes.

Configure npm trusted publishing for this repository before the first release. The publish workflow uses GitHub OIDC and does not use an NPM_TOKEN secret.

Local Development with `npm link`

If you are working from a local checkout of this repository and want a real qli command available in any directory:

cd /path/to/querylight-cli
npm install
npm run build
npm link

After that, you can use qli anywhere on your machine:

cd /some/project
qli --help

To remove the linked command later:

npm unlink -g @tryformation/querylight-cli

Quick Start

Initialize a workspace:

qli init

qli init creates the workspace config, enables dense and sparse retrieval for new workspaces, and pulls missing model assets when the runtime is available.

Add a local docs directory:

qli source add directory ./docs --name "Local Docs" --tag docs

Build the knowledge base:

qli ingest

Package it for deployment:

qli package ./docs-kb.zip

Search it:

qli search "API authentication"
qli search --workspace ./docs-kb.zip "API authentication"
qli search --source-type rss --since 2026-05-01 --has-publication-date
qli search-json '{"query":{"match":{"text":"API authentication"}},"size":5}'
curl -X POST http://127.0.0.1:3000/_simplesearch \
  -H 'content-type: application/json' \
  -d '{"query":"API authentication","topK":5}'
curl -X POST http://127.0.0.1:3000/_search \
  -H 'content-type: application/json' \
  -d '{"query":{"match":{"text":"API authentication"}},"size":5}'

Find related documents for an existing one:

qli related <document-id-or-uri>

Generate retrieval context:

qli context "How do I authenticate API requests?" --top-k 8

Serve the lexical index over HTTP:

qli serve

qli serve loads each knowledge base index once and reuses it for later requests. Use GET /_help to inspect routes, fields, and JSON DSL examples. Use GET /_knowledge_bases to list the mounted knowledge base prefixes. Use POST /_infer to produce dense and sparse query vectors from text. Use POST /_simplesearch for the same options as qli search. It defaults to hybrid retrieval and performs vector inference inside the request when vector indexes exist. Use POST /_search or POST /<configured-index-name>/_search for a single workspace. Use POST /<configured-index-name>/_simplesearch for simple search against a single workspace by configured index name. Use POST /<directory-name>/_search when --workspace points to a directory whose children are packaged .zip workspaces or directories that contain .kb. Use POST /<directory-name>/_simplesearch for simple search in multi-KB mode. Use POST /<directory-name>/_infer before vector searches in multi-KB mode. Packaged .zip knowledge bases are mounted read-only from the archive. qli serve does not extract them to workspace directories.

Example Skill: `qli` with `bunx` and `uv`

The repository includes an example skill for running qli without a global install and calling it from Python with uv:

examples/skills/qli-bunx-uv/SKILL.md

It covers:

running qli with bunx --bun @tryformation/querylight-cli
using --json for automation and agents
calling qli search and qli context from Python with subprocess

Example: Index `querylight.tryformation.com`

This example uses a local linked build of qli to create a test knowledge base for the Querylight documentation website.

Link the local CLI:

cd /path/to/querylight-cli
npm install
npm run build
npm link

Create a fresh test workspace:

mkdir -p ~/querylight-ts-search
cd ~/querylight-ts-search

Initialize the knowledge base:

qli init

Add the Querylight website as a source:

qli source add website https://querylight.tryformation.com \
  --name "Querylight TS Docs" \
  --max-depth 2 \
  --max-pages 50 \
  --include /docs/ \
  --tag docs

qli source add website may also detect one blog or news feed and register it as a separate rss source. Use --json when another tool needs the full list of created sources. Use qli source add page for one page. Use qli source add website when you want crawling or feed detection.

Ingest content and refresh the local index:

qli ingest

Inspect and query the result:

qli status
qli source list
qli search "BM25 ranking"
qli context "How does Querylight TS handle BM25 ranking?" --top-k 8

If you want the workspace somewhere else, use:

qli --workspace /custom/path/.kb <command>

Workspace

The default workspace is .kb/.

.kb/
  config.yaml
  sources/
    sources.jsonl
  documents/
    documents.jsonl
  chunks/
    chunks.jsonl
  raw/
  normalized/
  indexes/
    latest.json.gz
    latest.meta.json.gz
  runs/
  logs/

Vector model downloads are shared across workspaces under ~/.qli/models/ by default. qli init pulls missing model assets for enabled retrieval modes, so a new workspace is ready for vector indexing after setup.

Use a custom workspace with:

qli --workspace ./my-kb <command>

Package a built workspace with:

qli package ./docs-kb.zip --workspace ./my-kb

Read-only commands can use the zip directly:

qli search --workspace ./docs-kb.zip "authentication"
qli context --workspace ./docs-kb.zip "How does auth work?"
qli serve --workspace ./docs-kb.zip

Zip workspaces are read-only. Rebuild the directory workspace and package it again when source content changes.

Control the default remote concurrency in config.yaml:

crawler:
  maxConcurrentRequests: 5

Set crawl.maxConcurrentRequests on a website or RSS source when one source needs a different limit.

Control the default number of search results returned when --top-k is omitted:

search:
  defaultTopK: 50

For qli search --source-type rss with a time-window filter such as --since, --until, or --publication-date-from, qli uses 500 results when --top-k is omitted.

Supported Sources

Current source types:

file
directory
page
website
rss
markdown
text

Current local file ingestion support:

.md
.txt
.html
.htm
.pdf
.docx

Commands

All commands support:

--workspace <path>
--config <path>
--json
--silent
--verbose

--workspace accepts a directory workspace. Read-only commands also accept a packaged .zip workspace.

Long-running commands print progress to stderr by default. Use --silent to suppress progress output. Use --json when another tool needs stable structured output.

Initialize

qli init
qli init --workspace ./kb
qli init --force

Package

qli package ./docs-kb.zip
qli package ./deploy/docs-kb.zip --workspace ./docs/.kb
qli package ./docs-kb.zip --force --json

The archive stores workspace contents at the zip root. Use the zip with search, search-json, related, context, status, doctor, and serve.

Manage Sources

Add sources:

qli source add file ./docs/guide.md --name "Guide"
qli source add directory ./docs --name "Docs" --tag docs
qli source add page https://example.com/docs/auth --name "Auth Page"
qli source add website https://example.com --name "Example Site" --max-depth 2 --max-pages 50
qli source add website https://example.com --name "Example Site" --max-concurrent-requests 8
qli source add website https://example.com --name "Example Site" --json
qli source add rss https://example.com/feed.xml --name "Release Feed"
qli source add rss https://example.com/feed.xml --name "Release Feed" --max-concurrent-requests 3

page stores one page. website crawls a site and may detect one feed during registration.

Website sources may detect one blog or news feed during registration. When qli can infer a shared article prefix such as /blog/ or /news/, it adds that prefix to the website source excludes to reduce duplicate ingestion. Website and RSS sources default to 5 remote requests in flight per source. Override that in config.yaml or on the source.

List and manage them:

qli source list
qli source config <source-id> --retention-days 30
qli source config <source-id> --max-concurrent-requests 2
qli source config <source-id> --name "Docs Feed" --tag rss docs
qli source disable <source-id>
qli source enable <source-id>
qli source remove <source-id>

Find Related Documents

Build dense vectors first:

qli models pull --dense
qli rebuild

Or pull every model that is available on the current machine:

qli models pull

By default, qli models pull stores model assets in ~/.qli/models/ so multiple workspaces can reuse them.

Then ask for documents related to an existing document id or URI:

qli related <document-id>
qli related https://example.com/docs/auth

Ingest, Chunk, Index

qli ingest
qli chunk
qli index build
qli rebuild --silent

qli ingest fetches source content, updates affected chunks, and refreshes the index. Remote website and RSS fetches run concurrently. By default qli allows 5 in-flight requests per source.

Use qli rebuild when you want the explicit full pipeline command:

qli rebuild
qli rebuild --source <source-id>
qli rebuild --changed-only

Search and Retrieval

Search:

qli search "pricing API limits"
qli search "refund policy" --tag support --top-k 20
qli search --source-type rss,page --since 2026-05-01 --has-publication-date --top-k 25
qli search --source-name "Release Feed,Company Blog" --uri-prefix https://example.com/news,https://example.com/blog
qli search --source-type rss,page --top-k 25 --json
qli search "authentication" --json
qli search-json '{"query":{"bool":{"filter":[{"term":{"sourceType":"rss"}}]}},"size":10}' --json

Build retrieval context:

qli context "How do I configure the API?"
qli context "What changed in pricing?" --top-k 12 --max-chars 12000

Serve the lexical index over HTTP:

qli serve
qli serve --workspace ./docs/.kb --port 4000
qli serve --workspace ./docs-kb.zip --port 4000
qli serve --workspace ./kbs --host 0.0.0.0 --port 4000

For a single workspace, use POST /_simplesearch, POST /_search, or their /<configured-index-name>/... forms. For a directory of knowledge bases, use POST /<directory-name>/_simplesearch or POST /<directory-name>/_search. Child .zip files use the file stem as the route name. Use _simplesearch when the caller has text and wants the qli search workflow in one request. It accepts query, topK, source, sourceName, sourceType, uriPrefix, tag, metadata, date filters, retrieval, and showChunks. Use POST /_infer or POST /<directory-name>/_infer to turn query text into vectors before vector search. Use GET /_knowledge_bases to list the available prefixes before querying. Use GET /_help for route and request examples. The _search request body must be a Querylight JSON DSL object. Packaged .zip knowledge bases are mounted read-only from the archive. qli serve does not extract them to workspace directories. Index files are loaded once per knowledge base and reused across requests.

Build query vectors:

curl -s http://127.0.0.1:4000/_infer \
  -H 'content-type: application/json' \
  -d '{"text":"authentication flow","mode":"both"}'

Use the returned dense.vector with knn:

{
  "knn": {
    "field": "embedding",
    "vector": [0.12, -0.04, 0.98],
    "k": 10
  },
  "size": 10
}

Use the returned sparse.vector with sparse_vector or neural_sparse:

{
  "sparse_vector": {
    "field": "sparse",
    "vector": {
      "42": 0.91,
      "314": 0.62
    },
    "k": 10
  },
  "size": 10
}

Combine lexical, dense, and sparse retrieval with reciprocal rank fusion:

{
  "query": {
    "rrf": {
      "queries": [
        { "match": { "text": { "query": "authentication flow", "operator": "and" } } },
        { "knn": { "field": "embedding", "vector": [0.12, -0.04, 0.98], "k": 50 } },
        { "sparse_vector": { "field": "sparse", "vector": { "42": 0.91, "314": 0.62 }, "k": 50 } }
      ],
      "rank_constant": 20,
      "weights": [3, 1, 1]
    }
  },
  "size": 10
}

Change Inspection

qli diff
qli diff --source <source-id>
qli diff --document <document-id>
qli diff --since 2026-05-01

qli report changes --since 2026-05-01
qli report changes --source <source-id>

Workspace Inspection

qli status
qli doctor

JSON Output

Agent-facing and automation-friendly commands support --json.

The output envelope is:

{
  "ok": true,
  "command": "search",
  "workspace": "/absolute/path/.kb",
  "version": "0.1.0",
  "data": {}
}

Docker

Build the image:

docker build -t querylight-cli .

Run commands against a mounted workspace:

docker run --rm -v "$PWD:/data" querylight-cli init --workspace /data/.kb
docker run --rm -v "$PWD:/data" querylight-cli rebuild --workspace /data/.kb
docker run --rm -v "$PWD:/data" querylight-cli search --workspace /data/.kb "authentication"

Development

Install dependencies:

npm install

Run checks:

npm run check

Build:

npm run build