Endee LangChain Integration

A LangChain integration for the Endee vector database, enabling seamless vector storage and retrieval for RAG (Retrieval-Augmented Generation) applications.

Installation

npm install endee-langchain

Prerequisites

1. An Endee account and API key
2. An OpenAI API key (or another embeddings provider compatible with LangChain)
3. An existing Endee index (or create one programmatically)

Quick Start

1. Set Up Environment Variables

Create a .env file in your project root:

OPENAI_API_KEY=your-openai-api-key

2. Basic Usage

import { EndeeVectorStore } from "endee-langchain";
import { OpenAIEmbeddings } from "@langchain/openai";
import { Endee as EndeeClient } from "endee";
import { Document } from "@langchain/core/documents";

// Initialize embeddings
const embeddings = new OpenAIEmbeddings({
  model: "text-embedding-3-small",
  apiKey: process.env.OPENAI_API_KEY,
});

// Initialize Endee client and get index
const endeeClient = new EndeeClient("your-endee-api-key");
const index = await endeeClient.getIndex("your-index-name");

// Create vector store
const vectorStore = new EndeeVectorStore(embeddings, {
  endeeIndex: index,
});

// Add documents
const documents = [
  new Document({
    pageContent: "The powerhouse of the cell is the mitochondria",
    metadata: { source: "biology-textbook" },
  }),
  new Document({
    pageContent: "Buildings are made out of brick",
    metadata: { source: "architecture-guide" },
  }),
];

const ids = await vectorStore.addDocuments(documents);
console.log("Added documents with IDs:", ids);

// Alternative: Add texts directly
// const texts = ["Text 1", "Text 2"];
// const metadatas = [{ source: "doc1" }, { source: "doc2" }];
// const ids = await vectorStore.addTexts(texts, metadatas);

// Search for similar documents
const results = await vectorStore.similaritySearch("biology", 3);
console.log(results);

Creating an Index

If you need to create an index first:

const endeeClient = new EndeeClient("your-endee-api-key");

await endeeClient.createIndex({
  name: "my-index",
  dimension: 1536, // Must match your embedding model dimension
  spaceType: "cosine", // or "euclidean", "dotproduct"
  precision: "medium", // or "high", "low"
});

const index = await endeeClient.getIndex("my-index");

Static Factory Methods

fromTexts

Create a vector store from an array of text strings:

const texts = [
  "The powerhouse of the cell is the mitochondria",
  "Buildings are made out of brick",
  "Mitochondria are made out of lipids",
];

const metadatas = [
  { source: "doc1" },
  { source: "doc2" },
  { source: "doc3" },
];

const vectorStore = await EndeeVectorStore.fromTexts(
  texts,
  metadatas,
  embeddings,
  {
    endeeIndex: index,
  }
);

You can also use a single metadata object for all texts:

const vectorStore = await EndeeVectorStore.fromTexts(
  texts,
  { category: "science" }, // Single metadata object
  embeddings,
  { endeeIndex: index }
);

fromDocuments

Create a vector store from existing LangChain Documents:

const documents = [
  new Document({
    pageContent: "Document content here",
    metadata: { source: "example.com" },
  }),
  // ... more documents
];

const vectorStore = await EndeeVectorStore.fromDocuments(
  documents,
  embeddings,
  { endeeIndex: index }
);

API Reference

Constructor

new EndeeVectorStore(
  embeddings: EmbeddingsInterface,
  params: { endeeIndex: EndeeIndex }
)

Methods

addDocuments(documents, options?)

Add documents to the vector store. Documents are automatically embedded.

const ids = await vectorStore.addDocuments(documents, {
  ids: ["custom-id-1", "custom-id-2"], // Optional: provide custom IDs
});

addTexts(texts, metadatas?, options?)

Add texts to the vector store with optional metadatas. Similar to addDocuments but works with raw text strings.

// With array of metadatas
const texts = [
  "The powerhouse of the cell is the mitochondria",
  "Buildings are made out of brick",
  "Mitochondria are made out of lipids",
];

const metadatas = [
  { source: "doc1" },
  { source: "doc2" },
  { source: "doc3" },
];

const ids = await vectorStore.addTexts(texts, metadatas);

You can also use a single metadata object for all texts:

const ids = await vectorStore.addTexts(texts, { category: "science" });

Or provide custom IDs:

const ids = await vectorStore.addTexts(texts, metadatas, {
  ids: ["custom-id-1", "custom-id-2", "custom-id-3"],
});

addVectors(vectors, documents, options?)

Add pre-computed vectors and their corresponding documents.

const vectors = [[0.1, 0.2, ...], [0.3, 0.4, ...]];
const ids = await vectorStore.addVectors(vectors, documents);

similaritySearch(query, k?, filter?)

Search for similar documents by text query. Supports optional filtering.

// Basic search
const results = await vectorStore.similaritySearch("biology", 5);
// Returns: Document[]

// Search with filter
const filteredResults = await vectorStore.similaritySearch("biology", 5, [
  {type: {"$eq":"biology"}},
  {source: {"$eq":"https://example.com"}}
]);
// Returns: Document[]

similaritySearchWithScore(query, k?, filter?)

Search for similar documents and return similarity scores. Supports optional filtering.

// Basic search with scores
const results = await vectorStore.similaritySearchWithScore("biology", 5);
// Returns: [Document, number][]

// Search with filter and scores
const filteredResults = await vectorStore.similaritySearchWithScore("biology", 5, [
  { type: { "$eq": "biology" } }
]);
// Returns: [Document, number][]

delete({ ids?, filter? })

Delete vectors by IDs or by filter. You can use either ids or filter, but not both.

// Delete a single vector by ID
await vectorStore.delete({ ids: ["document-id"] });

// Delete multiple vectors by IDs
await vectorStore.delete({ ids: ["doc-1", "doc-2", "doc-3"] });

// Delete by filter (deletes all matching vectors)
await vectorStore.delete({ 
  filter: { 
    type: { "$eq": "biology" },
    source: { "$eq": "https://example.com" }
  } 
});

asRetriever(options?)

Convert the vector store into a LangChain retriever.

const retriever = vectorStore.asRetriever({
  k: 5, // Number of documents to retrieve
  searchType: "similarity", // or "mmr"
  searchKwargs: {
    // For MMR search
    fetchK: 20,
    lambda: 0.5,
  },
});

const results = await retriever.invoke("query text");

Advanced Usage

Using with LangChain Chains

import { RetrievalQAChain } from "langchain/chains";
import { OpenAI } from "@langchain/openai";

const retriever = vectorStore.asRetriever({ k: 5 });
const llm = new OpenAI({ temperature: 0 });

const chain = RetrievalQAChain.fromLLM(llm, retriever);

const answer = await chain.call({
  query: "What is the powerhouse of the cell?",
});
console.log(answer.text);

Batch Operations

The vector store automatically batches operations for efficiency. Documents are added in chunks of 100 by default.

Custom IDs

You can provide custom IDs when adding documents or texts:

// With documents
const documents = [/* ... */];
const customIds = ["doc-1", "doc-2", "doc-3"];
await vectorStore.addDocuments(documents, { ids: customIds });

// With texts
const texts = ["Text 1", "Text 2", "Text 3"];
const metadatas = [{ source: "doc1" }, { source: "doc2" }, { source: "doc3" }];
await vectorStore.addTexts(texts, metadatas, { ids: customIds });

Filtering

The vector store supports filtering in both search and delete operations. Filters are based on document metadata and use Endee's filtering operators.

For detailed information about filtering operators and syntax, see the Endee TypeScript SDK documentation.

Filtering Operators

| Operator | Description | Supported Type | Example | | -------- | ------------------------------------------------ | -------------- | ----------------------------------- | | $eq | Matches values that are equal | String, Number | { status: { "$eq": "published" } } | | $in | Matches any value in the provided list | String | { tags: { "$in": ["ai", "ml"] } } | | $range | Matches values between start and end (inclusive) | Number | { score: { "$range": [70, 95] } } |

Important Notes:

• Filters operate on fields provided under the filter key during upsert (document metadata)
• The $range operator supports values only within [0 – 999]. Normalize larger values before upserting
• In search methods, filters are passed as an array of filter objects

Filtering in Search

Use filters to narrow down search results based on metadata:

// Search with a single filter using $eq operator
const results = await vectorStore.similaritySearch("biology", 5, [
  { type: { "$eq": "biology" } }
]);

// Search with multiple filter conditions
const results = await vectorStore.similaritySearch("cell", 5, [
  { type: { "$eq": "biology" } },
  { source: { "$eq": "https://example.com" } },
  { category: { "$eq": "science" } }
]);

// Using $in operator to match any value in a list
const results = await vectorStore.similaritySearch("machine learning", 5, [
  { tags: { "$in": ["ai", "ml", "data-science"] } }
]);

// Using $range operator for numeric ranges
const results = await vectorStore.similaritySearch("high score", 5, [
  { score: { "$range": [70, 95] } }
]);

Filtering in Delete

Delete multiple vectors that match a filter:

// Delete all vectors with matching metadata using $eq
await vectorStore.delete({
  filter: {
    type: { "$eq": "archived" }
  }
});

// Delete using $in operator
await vectorStore.delete({
  filter: {
    tags: { "$in": ["deprecated", "old"] }
  }
});

Note: When using delete, you must provide either ids or filter, but not both.

Error Handling

The vector store includes error handling for common operations:

try {
  await vectorStore.addDocuments(documents);
} catch (error) {
  console.error("Failed to add documents:", error);
}

try {
  await vectorStore.delete({ ids: ["some-id"] });
} catch (error) {
  console.error("Failed to delete vector:", error);
}

Requirements

• Node.js 18+
• TypeScript 4.5+ (if using TypeScript)
• An active Endee account
• An embeddings provider (OpenAI, HuggingFace, etc.)

License

ISC

Author

Pankaj Singh (Endee Labs)

Support

For issues and questions, please open an issue on the GitHub repository.