@sf-bot/rag-indexes

RAG vector index helpers for HNSW and IVFFlat.

Overview

This package provides SQL functions for creating and managing pgvector indexes on RAG embeddings.

Functions

rag.create_hnsw_index(collection_id, model_name, m, ef_construction, distance_op)

Create an HNSW index for fast approximate nearest neighbor search.

-- Create HNSW index with defaults (m=16, ef_construction=64, cosine distance)
SELECT rag.create_hnsw_index('collection-uuid');

-- Create with custom parameters
SELECT rag.create_hnsw_index(
  'collection-uuid',
  'text-embedding-3-small',
  32,    -- m: connections per layer
  128,   -- ef_construction: index build quality
  'l2'   -- distance: cosine, l2, or ip
);

Parameters:

• m: Max connections per layer (default: 16). Higher = better recall, more memory
• ef_construction: Build-time quality (default: 64). Higher = better recall, slower build
• distance_op: Distance function - cosine, l2/euclidean, or ip/inner_product

rag.create_ivfflat_index(collection_id, model_name, lists, distance_op)

Create an IVFFlat index for fast indexing with good recall.

-- Create IVFFlat index with defaults (lists=100, cosine distance)
SELECT rag.create_ivfflat_index('collection-uuid');

-- Create with custom parameters
SELECT rag.create_ivfflat_index(
  'collection-uuid',
  'text-embedding-3-small',
  200,     -- lists: number of clusters
  'cosine' -- distance: cosine, l2, or ip
);

Note: IVFFlat requires data to be present before creating the index.

rag.list_vector_indexes(collection_id)

List all vector indexes, optionally filtered by collection.

SELECT * FROM rag.list_vector_indexes();
SELECT * FROM rag.list_vector_indexes('collection-uuid');

rag.drop_vector_index(index_name)

Drop a vector index by name.

SELECT rag.drop_vector_index('idx_rag_embedding_hnsw_...');

When to Use Which Index

| Index | Best For | Trade-offs | |-------|----------|------------| | HNSW | High recall, easy management | Slower inserts, more memory | | IVFFlat | Fast indexing, large datasets | Requires pre-populated data |

Dependencies

• @sf-bot/rag-core

License

SF License