raglite-toolkit

Build semantic search, multi-provider question answering, and REST APIs over your documents, directories, or web URLs in a few lines of TypeScript.

• PDF, TXT, JSON, Markdown, DOCX loaders out of the box
• Multi-document, directory, & URL ingestion — index folders, glob patterns, or web URLs with DocumentCollection
• Multi-provider LLMs — OpenAI, Anthropic (Claude), Google (Gemini), Mistral, Cohere, Groq, xAI (Grok), Ollama
• Multi-provider embeddings — OpenAI, Google, Mistral, Cohere, Voyage, Ollama, or a local sentence-transformer
• Cosine similarity scoring with L2-normalized vectors
• Content-hash cache invalidation — reindexes only when the file actually changes
• Per-document namespacing — indexes are isolated, so two documents never collide
• REST API via Hono with optional bearer-token auth
• Streaming answers
• TypeScript-first, ESM, strict types

Install

npm install raglite-toolkit

Provider SDKs are pulled in on demand. For local (offline) embeddings:

npm install @huggingface/transformers

Quick start

import { Document } from "raglite-toolkit";

const doc = new Document("./policy.pdf", {
  embeddings: { provider: "openai", apiKey: process.env.OPENAI_API_KEY },
  llm: { provider: "anthropic", apiKey: process.env.ANTHROPIC_API_KEY },
});

await doc.build();

const hits = await doc.search("refund policy", { topK: 3 });

const answer = await doc.ask("What is the refund policy?");
console.log(answer.text);

Multi-Document & Directory Ingestion (DocumentCollection)

Index entire directories (./docs), glob patterns, web URLs, or mixed file arrays seamlessly:

import { DocumentCollection } from "raglite-toolkit";

const collection = new DocumentCollection(["./docs", "https://example.com"], {
  embeddings: { provider: "local" },
  llm: { provider: "openai", apiKey: process.env.OPENAI_API_KEY },
});

// Concurrently index all documents in directory & web URLs
const result = await collection.build();
console.log(`Indexed ${result.totalDocuments} document(s), ${result.totalChunks} chunk(s).`);

// Search across all collection documents simultaneously
const hits = await collection.search("refund policy", { topK: 5 });

// Contextual Q&A across the entire collection
const answer = await collection.ask("What is the refund policy?");
console.log(answer.text);

Choose any LLM at ask-time

const openai = await doc.ask("Summarize this document", {
  llm: { provider: "openai", model: "gpt-4o", apiKey: process.env.OPENAI_API_KEY },
});

const claude = await doc.ask("Summarize this document", {
  llm: { provider: "anthropic", model: "claude-3-5-sonnet-20241022", apiKey: process.env.ANTHROPIC_API_KEY },
});

const gemini = await doc.ask("Summarize this document", {
  llm: { provider: "google", model: "gemini-2.0-flash", apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY },
});

const groq = await doc.ask("Summarize this document", {
  llm: { provider: "groq", model: "llama-3.3-70b-versatile", apiKey: process.env.GROQ_API_KEY },
});

const local = await doc.ask("Summarize this document", {
  llm: { provider: "ollama", model: "llama3.2", baseURL: "http://localhost:11434/api" },
});

Fully local with Ollama

const doc = new Document("./policy.pdf", {
  embeddings: {
    provider: "ollama",
    model: "embeddinggemma:latest",
    baseURL: "http://localhost:11434/api",
  },
  llm: {
    provider: "ollama",
    model: "gemma3:latest",
    baseURL: "http://localhost:11434/api",
  },
});

await doc.build();
console.log((await doc.ask("What is the refund policy?")).text);

Pluggable Vector Databases

By default, raglite uses a lightweight, in-memory vector store that persists indexes locally as JSON files. You can configure raglite to use a custom local or cloud-based vector database (like Qdrant, Pinecone, or LanceDB) by passing the vectorStore configuration option:

Memory Store (Default)

const doc = new Document("./policy.pdf", {
  vectorStore: { provider: "memory", storeDir: ".raglite" }
});

Qdrant (Local or Cloud)

Supports both local instances (e.g. running via Docker) and Qdrant Cloud clusters.

const doc = new Document("./policy.pdf", {
  vectorStore: {
    provider: "qdrant",
    url: "http://localhost:6333", // or Qdrant Cloud URL
    apiKey: "your-qdrant-api-key", // optional
    indexName: "my_collection"   // optional
  }
});

Pinecone (Cloud)

raglite automatically scopes document collections under Pinecone's native namespaces inside the index.

const doc = new Document("./policy.pdf", {
  vectorStore: {
    provider: "pinecone",
    url: "https://your-pinecone-index-host.svc.pinecone.io",
    apiKey: "your-pinecone-api-key"
  }
});

LanceDB (Local)

Runs a high-performance local database on the filesystem.

const doc = new Document("./policy.pdf", {
  vectorStore: {
    provider: "lancedb",
    storeDir: ".raglite"
  }
});

Custom Vector Store Class

You can also supply your own custom class instance directly as long as it implements the VectorStore interface:

import { VectorStore } from "raglite-toolkit";

class MyCustomStore implements VectorStore {
  // Implement methods: load, reset, add, search, count, saveIndexMetadata, readIndexMetadata
}

const doc = new Document("./policy.pdf", {
  vectorStore: new MyCustomStore()
});

Streaming

for await (const chunk of doc.askStream("Explain the introduction")) {
  process.stdout.write(chunk);
}

Local embeddings with @huggingface/transformers

const doc = new Document("./policy.pdf", {
  embeddings: { provider: "local", model: "Xenova/all-MiniLM-L6-v2" },
});
await doc.build();

REST API

const handle = await doc.serve({
  port: 8085,
  llm: { provider: "openai", apiKey: process.env.OPENAI_API_KEY },
  bearerToken: process.env.RAGLITE_TOKEN,
});
console.log(handle.url);

Endpoints:

| Method | Path | Description | | ------ | ---- | ----------- | | GET | /health | Liveness + index stats (auth-exempt) | | GET | /info | Configuration snapshot | | POST | /search | { query, topK?, scoreThreshold? } | | POST | /ask | { question, topK?, includeCitations?, stream? } |

curl http://127.0.0.1:8085/health

curl -X POST http://127.0.0.1:8085/search \
  -H 'authorization: Bearer <token>' \
  -H 'content-type: application/json' \
  -d '{"query":"refund policy","topK":3}'

curl -X POST http://127.0.0.1:8085/ask \
  -H 'authorization: Bearer <token>' \
  -H 'content-type: application/json' \
  -d '{"question":"What is the refund policy?","stream":true}'

CLI

raglite index ./policy.pdf --embed-provider openai --embed-key $OPENAI_API_KEY
raglite search ./docs "refund policy" --top-k 3
raglite ask ./docs "What is the refund policy?" \
  --llm-provider anthropic --llm-key $ANTHROPIC_API_KEY --stream
raglite serve https://example.com \
  --llm-provider openai --llm-key $OPENAI_API_KEY \
  --port 8085 --token $RAGLITE_TOKEN

Supported providers

LLMs

| Provider | Config key | Default model | | ---------- | ------------ | ------------- | | OpenAI | openai | gpt-4o-mini | | Anthropic | anthropic | claude-3-5-sonnet-20241022 | | Google | google | gemini-2.0-flash | | Mistral | mistral | mistral-large-latest | | Cohere | cohere | command-r-plus | | Groq | groq | llama-3.3-70b-versatile | | xAI (Grok) | xai | grok-2-latest | | Ollama | ollama | llama3.2 |

Embeddings

| Provider | Config key | Default model | | -------- | ---------- | ------------- | | OpenAI | openai | text-embedding-3-small | | Google | google | text-embedding-004 | | Mistral | mistral | mistral-embed | | Cohere | cohere | embed-english-v3.0 | | Voyage | voyage | voyage-3 | | Ollama | ollama | nomic-embed-text | | Local | local | Xenova/all-MiniLM-L6-v2 |

Configuration

new Document(path, {
  chunkSize: 500,       // words per chunk
  overlap: 50,          // words of overlap
  topK: 5,              // default results
  scoreThreshold: 0,    // cosine similarity floor (0..1)
  storeDir: ".raglite", // where indexes live
  embeddings: { provider: "local" },
  llm: { provider: "openai", apiKey: "..." },
  logLevel: "info",     // silent | info | debug
});

Development

npm install          # install dependencies
npm run typecheck    # tsc --noEmit
npm run lint         # biome lint
npm run format       # biome format --write
npm test             # vitest run
npm run verify       # typecheck + lint + tests (runs on prepublishOnly)
npm run build        # emit dist/

License

MIT