Red Sift Design System — RAG Server

An offline HTTP Retrieval-Augmented Generation server for the Red Sift Design System. Returns ranked, scored documentation chunks (components, props, patterns, demos, tokens) for any LLM, agent, or chatbot to consume.

Looking for IDE coding agents instead? See @redsift/ds-mcp-server — same data, but exposed over MCP/stdio for Copilot, Claude Code, Cursor, Windsurf. The two packages are siblings and intentionally complement each other. See Comparison.

Quick Start

npx @redsift/ds-rag-server
# → http://127.0.0.1:7345

In another terminal:

curl -X POST http://127.0.0.1:7345/retrieve \
  -H 'content-type: application/json' \
  -d '{"query":"data table with sorting"}'

You get back a ranked list of chunks (DataGrid will be in the top 3) with text, source metadata, and fused scores. Stuff them into a prompt for any LLM and you have a working RAG pipeline in ~10 lines of code.

Why This Exists

The Red Sift Design System publishes two AI-facing channels:

• MCP server (@redsift/ds-mcp-server) — speaks the Model Context Protocol over stdio. Built for IDE coding agents that need precise, structured tool calls (get_component_props, search_components).
• RAG server (this package) — speaks plain HTTP/JSON. Built for everything else: docs search boxes, Slack/web chatbots, internal support tools, custom agentic pipelines (LangGraph, Vercel AI SDK).

Use MCP when you want an IDE agent to generate correct component code. Use RAG when you want to answer fuzzy conceptual questions ("how do I show that something is loading?") in a chat surface.

Install Options

Public npm (recommended)

No .npmrc, no auth, no PAT:

npx @redsift/ds-rag-server

GitHub Packages (internal)

Same versions, requires a GitHub Personal Access Token with read:packages:

@redsift:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=${GITHUB_PAT}

CLI

npx @redsift/ds-rag-server [options]

Options:
  --port <port>      HTTP port (default 7345)
  --host <host>      bind host (default 127.0.0.1)
  --data-dir <path>  index/model directory (default: bundled data/)
  -h, --help         show help
  -v, --version      print version

HTTP API

POST /retrieve

Run hybrid retrieval (dense + BM25, RRF-fused) over the bundled corpus.

Request body:

{
  "query": "data table with sorting",
  "k": 8,
  "filters": {
    "kind": ["componentOverview", "componentProps"],
    "package": "@redsift/table",
    "componentName": "DataGrid"
  }
}

| Field | Type | Default | Notes | | ----------------------- | -------------------------------------------------------------------------------------------------- | ------- | ------------------------------------ | | query | string | — | required, non-empty | | k | integer 1–50 | 8 | number of results to return | | filters.kind | "componentOverview" \| "componentProps" \| "pattern" \| "demo" \| "token" \| "freeText" or array | — | restrict by chunk kind | | filters.package | string or string array | — | restrict by metadata.package | | filters.componentName | string or string array | — | restrict by metadata.componentName |

Response:

{
  "query": "data table with sorting",
  "count": 8,
  "results": [
    {
      "chunk": {
        "id": "componentOverview:components.json#@redsift/table/DataGrid",
        "kind": "componentOverview",
        "title": "DataGrid (@redsift/table)",
        "text": "# DataGrid\n\nA data grid …",
        "source": { "path": "components.json", "anchor": "@redsift/table/DataGrid" },
        "metadata": { "package": "@redsift/table", "componentName": "DataGrid" }
      },
      "score": 0.0476
    }
  ]
}

GET /healthz

{
  "status": "ok",
  "model": "Xenova/all-MiniLM-L6-v2",
  "dim": 384,
  "chunkCount": 1234,
  "indexedAt": "2026-05-06T10:03:18.593Z"
}

GET /chunks/:id

Returns a single chunk by id, or 404.

POST /embed (opt-in)

Returns a 384-dim embedding for an arbitrary string. Disabled by default; set ALLOW_EMBED=1 to expose.

Chunk Schema

type ChunkKind = 'componentOverview' | 'componentProps' | 'pattern' | 'demo' | 'token' | 'freeText';

interface Chunk {
  id: string;
  kind: ChunkKind;
  title: string;
  text: string;
  source: { path: string; anchor?: string };
  metadata: { package?: string; componentName?: string; tokenCategory?: string };
}

Environment Variables

| Variable | Default | Notes | | ------------- | --------------- | ---------------------------------------- | | PORT | 7345 | overridden by --port | | HOST | 127.0.0.1 | overridden by --host | | DATA_DIR | bundled data/ | overridden by --data-dir | | CORS_ORIGIN | unset (off) | comma-separated origins or * to opt-in | | ALLOW_EMBED | unset | set to 1 to expose POST /embed |

Integration Recipes

Three copy-pasteable patterns under docs/integrations/:

• Plain fetch / single-shot RAG — one /retrieve, stuff into prompt, call any LLM.
• Vercel AI SDK tool — wrap /retrieve as a tool({ execute }).
• LangGraph agentic RAG — copy-pasteable LangGraph snippet with grading + query rewriting.

How It Works

1. Corpus — components, patterns, demos, tokens, and an llms-full.txt fallback are extracted from the design system at publish time and bundled into data/source/ inside the package.
2. Chunking — deterministic (re-runs over the same source produce identical ids).
3. Dense index — every chunk is embedded with the bundled Xenova/all-MiniLM-L6-v2 ONNX model (384-dim, ~25 MB, MIT). Vectors are L2-normalised so cosine similarity = dot product. Stored as a flat Float32Array in data/index.bin.
4. Keyword index — a MiniSearch BM25 index over { title, text, componentName, package } in data/keyword.json.
5. Retrieval — at query time, embed once, take top-30 dense and top-30 keyword, fuse via Reciprocal Rank Fusion (RRF, k=60), apply filters, return top-k.
6. Offline — env.allowRemoteModels = false. The package is a sealed offline artefact: no HuggingFace fetch, no API key, no network at runtime.

The published tarball is ~40 MB (model + index + source corpus). Built and verified by the publish-rag workflow with provenance attestations.

Local Development

yarn rag:bundle  # build data/source/, index.bin, keyword.json, manifest.json, model/
yarn rag:dev     # start the server with tsx (no build step)
yarn rag:build   # tsc → dist/
yarn rag:start   # node dist/index.js

Security Model

The server is designed to be a localhost-or-private-network process. There is no built-in authentication, rate limiting, or multi-tenant isolation. If you expose it externally, put a reverse proxy with auth in front of it. CORS is off by default; set CORS_ORIGIN to opt in.

License

MIT.