@se-studio/search

AI-powered site search for Next.js marketing sites using Upstash Search. Combines semantic and full-text search with zero infrastructure to manage.

Overview

This package provides:

• Search client – typed wrapper around @upstash/search with automatic batch handling
• Content indexing – extracts searchable text from CMS content using MarkdownConverter
• Webhook handler – incremental index updates on Contentful publish/delete
• Full rebuild – enumerates all content and re-indexes in batches
• API route factories – drop-in Next.js route handlers for search and rebuild
• Client hook – useSearch() for building search UIs with debouncing

Setup

1. Install

The package is a workspace dependency. Add it to your app's package.json:

{
  "dependencies": {
    "@se-studio/search": "workspace:*"
  }
}

2. Create an Upstash Search database

Go to console.upstash.com/search and create a database. Copy the REST URL and token.

3. Environment variables

Add to your .env.local:

UPSTASH_SEARCH_REST_URL=https://your-search-url.upstash.io
UPSTASH_SEARCH_REST_TOKEN=your-token

4. Search config

Create src/lib/search-config.ts:

import 'server-only';
import type { SearchIndexingConfig } from '@se-studio/search';

export const searchIndexingConfig: SearchIndexingConfig = {
  searchIndex: {
    connection: {
      url: process.env.UPSTASH_SEARCH_REST_URL ?? '',
      token: process.env.UPSTASH_SEARCH_REST_TOKEN ?? '',
    },
    publishedIndexName: 'published',
    previewIndexName: 'preview',
  },
  contentTypes: [
    { type: 'page', enabled: true },
    { type: 'article', enabled: true },
    { type: 'person', enabled: false },
  ],
  indexComponents: true,
  respectIndexedFlag: true,
  respectHiddenFlag: true,
};

5. Search API route

Create src/app/api/search/route.ts:

import { createSearchApiHandler } from '@se-studio/search/api';
import { buildInformation } from '@/lib/converter-context';
import { searchIndexingConfig } from '@/lib/search-config';

export const GET = createSearchApiHandler({
  searchConfig: searchIndexingConfig.searchIndex.connection,
  publishedIndexName: searchIndexingConfig.searchIndex.publishedIndexName,
  previewIndexName: searchIndexingConfig.searchIndex.previewIndexName,
  isPreview: buildInformation.preview ?? false,
});

6. Rebuild API route

Create src/app/api/search/rebuild/route.ts:

import { createSearchClient } from '@se-studio/search/client';
import { rebuildSearchIndex } from '@se-studio/search/indexing';
import { createRebuildApiHandler } from '@se-studio/search/api';
import { buildOptions, getContentfulConfig } from '@/lib/cms-server';
import { customerName, license } from '@/lib/constants';
import { buildInformation, converterContext } from '@/lib/converter-context';
import { searchIndexingConfig } from '@/lib/search-config';
import { baseUrl, revalidationSecret } from '@/lib/server-config';

const isPreview = buildInformation.preview ?? false;

export const POST = createRebuildApiHandler({
  rebuildSecret: revalidationSecret ?? '',
  isPreview,
  rebuildFn: () => {
    const config = getContentfulConfig(isPreview);
    const client = createSearchClient(searchIndexingConfig.searchIndex.connection);
    const indexName = isPreview
      ? searchIndexingConfig.searchIndex.previewIndexName
      : searchIndexingConfig.searchIndex.publishedIndexName;

    return rebuildSearchIndex({
      client,
      indexName,
      indexingConfig: searchIndexingConfig,
      converterContext,
      contentfulConfig: config,
      fetchOptions: buildOptions({ preview: isPreview }),
      urlCalculators: converterContext.urlCalculators,
      siteConfig: { canonicalBaseUrl: baseUrl, source: customerName, license },
    });
  },
});

7. Webhook integration

Update your src/app/api/revalidate/route.ts to call the search webhook handler after cache revalidation. See the example-brightline app for the full pattern.

8. Client-side search

'use client';
import { useSearch } from '@se-studio/search/hooks';

export function SearchPage() {
  const { query, setQuery, results, isLoading, error, totalCount } = useSearch();

  return (
    <div>
      <input value={query} onChange={(e) => setQuery(e.target.value)} placeholder="Search..." />
      {isLoading && <p>Searching...</p>}
      {error && <p>Error: {error}</p>}
      {results.map((r) => (
        <a key={r.id} href={r.metadata.href}>
          <h3>{r.content.title}</h3>
          <p>{r.content.description}</p>
        </a>
      ))}
    </div>
  );
}

Triggering a full rebuild

curl -X POST https://your-site.com/api/search/rebuild \
  -H "Authorization: Bearer YOUR_REVALIDATION_SECRET"

The index to rebuild (published vs preview) is determined by the app's own isPreview flag set at initialization time in createRebuildApiHandler.

Advanced: documentTransformer

SearchIndexingConfig accepts an optional documentTransformer callback that is invoked for every SearchDocument after it has been built — in both the full-rebuild and webhook-driven incremental-update paths. Use it to patch metadata, inject custom fields, or drop specific documents entirely.

import type { ContentData } from '@se-studio/search'; // re-exported from @se-studio/markdown-renderer
import type { SearchIndexingConfig } from '@se-studio/search';

export const searchIndexingConfig: SearchIndexingConfig = {
  // ...
  documentTransformer: (doc, contentData) => {
    // Augment: add a custom field
    doc.metadata.myCustomField = 'value';

    // Drop: returning null removes the document from the index
    if (doc.metadata.slug === 'draft-preview') return null;

    return doc;
  },
};

Signature:

documentTransformer?: (doc: SearchDocument, contentData: ContentData) => SearchDocument | null;

• doc – the built SearchDocument (safe to mutate in place or return a new object)
• contentData – the raw CMS content data for the entry, giving access to all fields
• Return the document (modified or not) to include it, or null to drop it

The transformer runs on every chunk of a multi-chunk entry, so a 3-chunk page will call it 3 times.

Architecture

• Single Upstash database, two indexes: published and preview
• Text extraction: Uses MarkdownConverter to deeply extract text from page components, then strips markdown formatting
• Content truncation: Body text is truncated to ~4,000 chars (Upstash limit)
• Batch upsert: Documents are upserted in batches of 100 (Upstash API limit)
• Webhook-driven: Incremental updates on Contentful publish/delete events
• Flags: Respects indexed and hidden fields on content entries

Subpath Exports

| Import | Purpose | |--------|---------| | @se-studio/search | Types only | | @se-studio/search/client | createSearchClient() | | @se-studio/search/indexing | rebuildSearchIndex(), buildSearchDocument() | | @se-studio/search/webhook | createSearchWebhookHandler() | | @se-studio/search/api | createSearchApiHandler(), createRebuildApiHandler() | | @se-studio/search/hooks | useSearch() |