@kognitivedev/rag

RAG pipeline with 6 chunkers, 3 vector stores, and AI SDK tool output.

Installation

bun add @kognitivedev/rag ai zod

Quick Start

import { DocumentPipeline, RecursiveTextChunker, InMemoryVectorStore, AISDKEmbeddingProvider } from "@kognitivedev/rag";
import { openai } from "@ai-sdk/openai";

const pipeline = new DocumentPipeline({
  chunker: new RecursiveTextChunker({ chunkSize: 1000 }),
  embedder: new AISDKEmbeddingProvider({ model: openai.embedding("text-embedding-3-small") }),
  vectorStore: new InMemoryVectorStore(),
});

await pipeline.ingest([{ content: "Your documents here" }]);
const results = await pipeline.search("query", { topK: 5 });
const tool = pipeline.asAISDKTool(); // Use in streamText

For PDFs, DOCX, images, and OCR-backed ingestion, preprocess files with @kognitivedev/documents and then pass the resulting text documents into pipeline.ingest().

Chunkers

RecursiveTextChunker, MarkdownChunker, TokenChunker, HTMLChunker, JSONChunker, SentenceChunker

Vector Stores

InMemoryVectorStore (dev), PgVectorStore (PostgreSQL), QdrantVectorStore (Qdrant)