Strapi plugin strapi-plugin-elasticsearch

A plugin to enable integrating Elasticsearch with Strapi CMS.

Installation

via npm:

npm i @geeky-biz/strapi-plugin-elasticsearch

via yarn:

yarn add @geeky-biz/strapi-plugin-elasticsearch

Plugin Configuration

Within your Strapi project's config/plugin.js, enable the plugin and provide the configuration details:

module.exports = {
    // ...
    'elasticsearch': {
      enabled: true,
      config: {
        indexingCronSchedule: "<cron schedule>",
        searchConnector: {
          host: "<hostname for Elasticsearch>",
          username: "<username for Elasticsearch>",
          password: "<password for Elasticsearch>",
          certificate: "<path to the certificate required to connect to Elasticsearch>"
        },
        indexAliasName: "<Alias name for the Elasticsearch index>"
      }  
    },
    // ...
  }

Example plugin configuration (with adequate .env variables set up):

module.exports = {
    // ...
    'elasticsearch': {
      enabled: true,
      config: {
        indexingCronSchedule: "00 23 * * *", //run daily at 11:00 PM
        searchConnector: {
          host: process.env.ELASTIC_HOST,
          username: process.env.ELASTIC_USERNAME,
          password: process.env.ELASTIC_PASSWORD,
          certificate: path.join(__dirname, process.env.ELASTIC_CERT_NAME)
        },
        indexAliasName: process.env.ELASTIC_ALIAS_NAME
      }  
    },
    // ...
  }

Ensuring connection to Elasticsearch

When connected to Elasticsearch, the Connected field within the Setup Information screen shall display true.

Configuring collections & attributes to be indexed

The Configure Collections view displays the collections and the fields setup to be indexed.

From this view, individual collection can be selected to modify configuration:

Configuring indexing for dynamic zone or component attributes

To enable indexing content for attributes of type component or dynamiczone, additional information needs to be provided via JSON in the following format:

{
  "subfields": [
        {
          "component": "<component name within schema.json>",
          "field": "<field name from within that component>"
        },
        {...},
        {...}
      ]
}

Example 1:

If we have an attribute called seo_details of type component like the following within our collection schema.json:

    "seo_details": {
      "type": "component",
      "repeatable": false,
      "component": "metainfo.seo"
    },

And, if we seek to index the contents of the meta_description field belonging to the component seo, our subfields configuration should be:

{
  "subfields": [
        {
          "component": "metainfo.seo",
          "field": "meta_description"
        }
      ]
}

Example 2:

If we have an attribute called sections of type dynamiczone within our collection schema.json:

    "sections": {
      "type": "dynamiczone",
      "components": [
        "content.footer",
        "content.paragraph",
        "content.separator",
        "content.heading"
      ]
    },
...

And, if we seek to index the contents of the fields title for content.heading and details as well as subtext for content.paragraph, our subfields configuration should be:

{
  "subfields": [
        {
          "component": "content.paragraph",
          "field": "details"
        },
        {
          "component": "content.paragraph",
          "field": "subtext"
        },
        {
          "component": "content.heading",
          "field": "title"
        }
      ]
}

The subfields JSON also supports multiple level of nesting:

{
  "subfields": [
        {
          "component": "content.footer",
          "field": "footer_link",
          "subfields": [
            {
              "component": "content.link",
              "field": "display_text"
            }
          ]
        }
      ]
}

Note: Indexing of relations attributes isn't yet supported.

Exporting and Importing indexing configuration

To enable backing up the indexing configuration or transferring it between various environments, these can be Exported / Imported from the Configure Collections view.

Scheduling Indexing

Once the collection attributes are configured for indexing, any changes to the respective collections & attributes is marked for indexing. The cron job (configured via indexingCronSchedule) makes actual indexing requests to the connected Elasticsearch instance.

• Trigger Indexing triggers the cron job immediately to perform the pending indexing tasks without waiting for the next scheduled run.
• Rebuild Indexing completely rebuilds the index. It may be used if the Elasticsearch appears to be out of sync from the data within Strapi.

Whenever a collection is configured for indexing, it may already have data that needs to be indexed. To facilitate indexing of the past data, a collection can be scheduled for indexing in the next cron run from the Configure Collections view:

Searching

You may directly use the Elasticsearch search API or you may use the Search API exposed by the plugin (at /api/elasticsearch/search). The plugin search API is just a wrapper around the Elasticsearch search API that passes the query parameter to the Elasticsearch search API and returns the results coming from Elasticsearch:

For example, the below API call would result into the Elasticsearch search API being triggered with the query

`/api/elasticsearch/search?query=query%5Bbool%5D%5Bshould%5D%5B0%5D%5Bmatch%5D%5Bcity%5D=atlanta`

would result into the Elasticsearch search API being triggered with query

query[bool][should][0][match][city]=atlanta

The plugin's API would return the response from the Elasticsearch search API.

Note: To use the search API (at /api/elasticsearch/search), you will have to provide access via Settings -> Users & Permissions Plugin -> Roles -> (Select adequate role) -> Elasticsearch -> search.

Extending Search API

The recommended was to enhance the Search API is to write your own route and controller. Below is an example of how this can be achieved (this example adds pagination capability to the search API):

• Within your setup, create src/extensions/elasticsearch/strapi-server.js with the following contents:

const { Client } = require('@elastic/elasticsearch')
const qs = require('qs');

let client = null;

module.exports = (plugin) => {

    client = new Client({
        node: plugin.config.searchConnector.host,
        auth: {
          username: plugin.config.searchConnector.username,
          password: plugin.config.searchConnector.password
        },
        tls: {
          ca: plugin.config.searchConnector.certificate,
          rejectUnauthorized: false
        }
    });

    plugin.controllers['performSearch'].enhancedSearch = async (ctx) => {
            try
            {
                const params = qs.parse(ctx.request.query)
                const query = params.search;
                const pagesize = params.pagesize;
                const from = params.from;
                const result= await client.search({
                    index: plugin.config.indexAliasName,
                    query: { "bool" : { "should" : [ { "match": { "content": "dummy"} } ] } },
                    size: pagesize,
                    from: from 
                });
                return result;
            }
            catch(err)
            {
                console.log('Search : elasticClient.enhancedSearch : Error encountered while making a search request to ElasticSearch.')
                throw err;
            }
    }

    plugin.routes['search'].routes.push({
        method: 'GET',
        path: '/enhanced-search',
        handler: 'performSearch.enhancedSearch',
      });
  
    
    return plugin;
};

• This will create a new route /api/elasticsearch/enhanced-search being served by the function defined above.
• You can add / modify the routes and controllers as necessary.

Virtual Collections

Virtual Collections allow you to index and search data that does not directly map to a single Strapi collection type. This is useful for aggregating, transforming, or combining data from multiple sources before indexing it in Elasticsearch.

What is a Virtual Collection?

A virtual collection is a logical grouping of data that you define, which can be indexed into Elasticsearch as if it were a regular Strapi collection. You control how the data is extracted, transformed, and indexed.

For example -
Let's say that you have two collections defined in Strapi:

• Restaurant
• Chain

• Each chain is related to one or more restaurants. Some restaurants don't have a chain.

Now let's say you want to index the restaurants, but in each restaurant you'd like to include some information about the restaurant, and you also want to merge some fields (e.g: if a Restaurant doesn't have a description - it inherits it from its chain ).
To do that, you need to create a virtual-collection in this plugin configuration, based on your existing restaurant collection.

How to Register a Virtual Collection

To register a virtual collection, you need to provide a configuration object that implements the VirtualCollectionConfig interface. This is typically done in your plugin or project code.

Example:

// src/extensions/elasticsearch/virtual-collections/my-virtual-collection.ts

import { VirtualCollectionConfig } from 'strapi-plugin-elasticsearch/server/types/virtual-collections.type';

const myVirtualCollection: VirtualCollectionConfig = {
  collectionName: 'virtual::my-virtual-collection',
  indexAlias: 'my_virtual_collection_index',
  extractData: async (page, pageSize) => {
    // Fetch and return an array of entities for the given page
    // Example: aggregate data from multiple collections
    return [];
  },
  extractByIds: async (ids) => {
    // Fetch and return entities by their IDs
    return [];
  },
  getIndexItemId: (itemId, collectionName) => `${collectionName}::${itemId}`,
  triggers: [
    {
      collection: 'api::some-collection.some-collection',
      getIdsToReindex: (event) => {
        // Return an array of virtual collection item IDs to reindex
        return [event.result.id];
      },
    },
  ],
  mappings: {
    // Optional: Elasticsearch mappings for this collection
  },
};

export default myVirtualCollection;

Key Properties

• collectionName: Unique name for your virtual collection.
• indexAlias: (Optional) Alias for the Elasticsearch index.
• extractData(page, pageSize): Function to fetch paginated data for indexing.
• extractByIds(ids): Function to fetch specific items by ID.
• getIndexItemId(itemId, collectionName): (Optional) Function to generate unique Elasticsearch document IDs.
• triggers: Array of triggers that specify which Strapi collections should cause this virtual collection to reindex.
• mappings: (Optional) Elasticsearch mappings for the index.

How Triggers Work

Each trigger listens to changes on a specified Strapi collection. When a change occurs, the getIdsToReindex function is called with the event data, and should return the IDs of the virtual collection items that need to be reindexed.
These items are then reindexed - they are fetched using extractByIds and are sent to ElasticSearch. If an item cannot be found using extractByIds - it is deleted from ElasticSearch.

Registering the Virtual Collection

Register your virtual collection by specifying the virtualCollections property in this plugin's configuration, in config/plugins.js.

Example:

// config/plugins.js

import myVirtualCollection from './virtual-collections/my-virtual-collection';

module.exports = async ({ env }) => {
  return {
    elasticsearch :{
      enabled: true,
      config: {
        ...
        virtualCollections: [
          myVirtualCollection,
        ]
      }
    }
  }
};

Indexing and Searching

Once registered, your virtual collection will be indexed according to the triggers and can be searched like any other indexed collection in Elasticsearch.

Bugs

For any bugs, please create an issue here.

About

• This plugin is created by Punit Sethi.
• I'm an independent developer working on Strapi migrations, customizations, configuration projects (see here).
• For any Strapi implementation requirement, write to me at [email protected].