🧠 knowy

knowy is a lightweight, high-performance RAG (Retrieval-Augmented Generation) engine built on LanceDB and HuggingFace Transformers. It simplifies the process of embedding, storing, and retrieving knowledge with support for scoped knowledge bases, custom metadata, and precise text splitting.

🚀 Features

• Hybrid API: Use it globally for quick actions or scoped for organized KB management.
• Smart Splitting: Recursive text splitter that preserves sentence integrity and prevents "semantic dilution."
• Flexible Metadata: Store any extra data (user IDs, versions, tags) and filter results using SQL-like queries.
• Automatic Timestamps: Every record is stamped with __sys_created_at_ for easy time-based filtering.
• Local-First: Powered by ONNX-optimized embeddings and LanceDB for lightning-fast local vector search.

📦 Installation

npm install knowy

🛠️ Usage

Initialization

import { knowy } from 'knowy';

const kbs = await knowy("./my_knowledge_db");

Adding Knowledge

You can use the Global API or the Scoped API:

// Global Style
await kbs.addText("hr", "benefits", "Unlimited vacation policy.", "manual.pdf", { version: 1.2 });

// Scoped Style (Recommended for clean code)
const legal = kbs("legal");
await legal.ingest("privacy", longDocumentText, "privacy_policy.txt", {
  chunkSize: 250,
  overlap: 50,
  metadata: { classification: "confidential" }
});

Advanced Searching & Filtering

Retrieve the most relevant context while filtering by your custom metadata:

// Search a specific KB with a metadata filter
const results = await kbs("legal").search("What is the retention policy?", {
  where: "classification = 'confidential' AND __sys_created_at_ > 1709400000000",
  limit: 3
});

console.log(results[0].content.text);
console.log(results[0].metadata.classification); // "confidential"

Management

// List all Knowledge Bases
const list = await kbs.list();

// Delete a KB
await kbs.delete("temp_data");

⚙️ Configuration

| Option | Default | Description | | :--- | :--- | :--- | | chunkSize | 250 | Maximum characters per chunk. Lower values increase precision. | | overlap | 80 | Character overlap between chunks to prevent splitting keywords. | | where | NULL | A SQL-string for metadata filtering (e.g., "user_id = 5"). |

🏗️ Architecture

knowy uses a Recursive Splitter to ensure that data is chunked logically at paragraph and sentence boundaries. These chunks are converted into vectors using the Qwen3-Embedding-0.6B-ONNX model, providing a perfect balance between speed and semantic accuracy for local environments.