@nexo-labs/payload-typesense

A powerful, production-ready search plugin that integrates Typesense with Payload CMS. This plugin provides lightning-fast, typo-tolerant search capabilities with real-time synchronization, and a comprehensive RAG (Retrieval Augmented Generation) system for building AI-powered conversational agents.

Features

• Real-time Sync: Automatically syncs your Payload collections to Typesense.
• Flexible Indexing: Map a single Payload collection to multiple Typesense indices (tables) with different configurations.
• Advanced Search:
  • Semantic Search: Vector-based search using embeddings.
  • Keyword Search: Traditional text matching with typo tolerance.
  • Hybrid Search: Combines vector and keyword search for optimal relevance.
• RAG & AI Agents:
  • Built-in support for conversational AI agents.
  • Integrates with OpenAI and Gemini.
  • Manages chat sessions and history.
  • Streaming responses (SSE).
• Optimized Performance: Configurable HNSW parameters for vector search and batch processing for sync.

Installation

pnpm add @nexo-labs/payload-typesense

Configuration

The plugin is configured in your payload.config.ts.

Basic Setup

import { buildConfig } from 'payload/config';
import { typesenseSearch } from '@nexo-labs/payload-typesense';

export default buildConfig({
  // ...
  plugins: [
    typesenseSearch({
      typesense: {
        apiKey: process.env.TYPESENSE_API_KEY,
        nodes: [
          {
            host: process.env.TYPESENSE_HOST,
            port: 443,
            protocol: 'https',
          },
        ],
      },
      features: {
        sync: { enabled: true },
        search: { enabled: true },
        embedding: {
          type: 'openai',
          apiKey: process.env.OPENAI_API_KEY,
          model: 'text-embedding-3-small',
        },
      },
      collections: {
        posts: [
          {
            enabled: true,
            tableSuffix: 'v1',
            searchFields: ['title', 'content'],
            chunkingStrategy: { type: 'markdown' },
          },
        ],
      },
    }),
  ],
});

Sync Configuration

You can define how each collection is synced to Typesense. The collections object maps collection slugs to an array of CollectionTableConfig.

collections: {
  posts: [
    {
      enabled: true,
      // Optional suffix for the Typesense collection name
      // Result: "posts_summary"
      tableSuffix: 'summary',
      // Fields to include in the Typesense document
      searchFields: ['title', 'excerpt'],
      // Chunking strategy for long text
      chunkingStrategy: { type: 'simple', chunkSize: 500 },
    },
    {
      enabled: true,
      // Explicitly naming the table overrides the default naming convention
      // Result: "my_custom_posts_index"
      tableName: 'my_custom_posts_index',
      searchFields: ['title', 'content'],
      chunkingStrategy: { type: 'markdown' },
    }
  ]
}

RAG & Agents Configuration

Enable RAG to build chat interfaces on top of your data.

features: {
  rag: {
    enabled: true,
    agents: [
      {
        slug: 'support-bot',
        name: 'Support Assistant',
        systemPrompt: 'You are a helpful support assistant.',
        llmModel: 'gpt-4o-mini',
        // Collections this agent can access
        searchCollections: ['posts_full_text'],
      }
    ]
  }
}

Usage

Search API

The plugin exposes endpoints for searching your data.

Endpoint: /api/typesense/search

// Example using fetch
const response = await fetch('/api/typesense/search', {
  method: 'POST',
  body: JSON.stringify({
    query: 'how to install',
    collections: ['posts_full_text'],
  })
});

RAG Chat API

Endpoint: /api/typesense/rag/chat

This endpoint supports Server-Sent Events (SSE) for streaming AI responses.

// Example client-side code (simplified)
const eventSource = new EventSource('/api/typesense/rag/chat?message=hello');

eventSource.onmessage = (event) => {
  const data = JSON.parse(event.data);
  if (data.type === 'token') {
    console.log(data.data); // Streamed text
  }
};

Roadmap

We are actively working to improve the modularity and flexibility of the plugin.

1. Modular Embedding Strategies (High Priority)

Goal: Allow different embedding models for different collections (tables).

• Current State: Global embedding configuration only.
• Future:
  • Move embedding config to CollectionTableConfig.
  • Support "Table X" -> Gemini, "Table Y" -> OpenAI Large.
  • Update RAG agents to dynamically select the correct embedding model based on the collection being queried.

2. Expanded Provider Support

• Embeddings: Add support for Cohere, HuggingFace, and local models (via Ollama).
• LLMs: Add support for Anthropic (Claude), Mistral, and local LLMs.

4. Admin UI

• Dashboard within Payload Admin to view Typesense collection status.
• Buttons to manually trigger re-sync or re-indexing.
• Visual management of RAG agents.

5. Advanced Sync Controls

• Soft Deletes: Option to mark documents as deleted in Typesense instead of removing them.
• Conditional Sync: specialized syncCondition function to control which documents are synced based on their content (e.g., only sync published posts).