Dataweavers - Next.js Redis Caching

Provides a standardised Redis caching integration for Next.js applications. On install, the package automatically copies the necessary source files into your project, registers the cache handler in your instrumentation setup, and updates your .env file with the required configuration variables.

Internally it uses @fortedigital/nextjs-cache-handler as the underlying cache handler.

What this is

A Redis caching solution that can be used with the Dataweavers Arc platform with extensibility and alteration points to get the most out of the platform / application.

What this isn't

A one solution fits all use cases. There are other caching packages available and other caching mechanisms for your next.js application. This is an implementation to enable your application to utilise the Redis resource for page caching in next.js.

Features

• Redis cache handler – A handler that writes to Redis for next.js standard caching functionality, i.e. pages.
• Cache clear API route – A secured POST /api/clear-page-cache endpoint that flushes all Redis keys added by this module.
• Cache populate API route – An example POST /api/populate-page-cache endpoint that batch-revalidates pages. The example extends the patch processing by walking the Sitecore sitemap functionality. This file is intended to be customised for your project.
• Instrumentation registration – Automatically registers the cache handler initialisation in your instrumentation.ts / instrumentation-initialiser.js so the cache is pre-populated on startup when configured. This OR the populate-page-cache API is to be used for cache prepopulation.

Installation

1. Install the package

npm install @dataweavers/redis-page-caching

The postinstall script runs automatically and copies the following into your project:

• src/pages/api/clear-page-cache.ts or src/app/api/clear-page-cache.ts
• src/pages/api/populate-page-cache.ts or src/app/api/populate-page-cache.ts

It also appends the required environment variable(s) to your .env file and registers the cache initialisation in your instrumentation.ts.

2. Manually set the next.js cache handler in the next.config.js file as per below.

//set the expected Redis cache handler path for the platform to use when caching responses
cacheHandler: path.join(process.cwd(),'node_modules/@dataweavers/nextjs-redis-caching/dist/cache-handler.mjs'),
cacheMaxMemorySize: 0, // disable default in-memory caching

3. Set the environment variables as required (below) to connect Redis (REDIS_URL and REDIS_ACCESS_KEY are likely required, other values are optional).

Configuration

The following environment variables are added to your .env file on install:

| Variable | Required | Default | Description | |---|---|---|---| | REDIS_URL | Yes | (empty) | Connection URL for the Redis instance, e.g. redis://localhost:6379. Leave blank for local development to fall back to LRU. | | REDIS_ACCESS_KEY | No | (empty) | Access key / password for the Redis instance (e.g. Azure Cache for Redis). | | REDIS_KEY_PREFIX | No | nextjs: | Prefix applied to all Redis keys. Useful when sharing a Redis instance across multiple applications or environments. | | REDIS_CACHE_INIT | No | manual | Controls how the cache is pre-populated on startup. See Cache initialisation modes. | | CACHE_MANAGEMENT_SECRET | Yes | (must be set) | Secret key used to authenticate requests to the cache management API routes (/api/clear-cache, /api/populate-cache). Use a strong random string. | | CACHE_POPULATE_BATCH_SIZE | No | 25 | Number of pages processed per batch during cache population. | | CACHE_POPULATE_BATCH_DELAY_MS | No | 5000 | Delay in milliseconds between batches during cache population. | | DATAWEAVERS_LOG_LEVEL | No | error | Log verbosity. Options: silent, error, warn, info, debug, trace. |

Dataweavers recommends that you store the secrets/keys in a secure vault rather than a variable file.

Contents - copy files

1. Cache clear API route

The copied api/clear-page-cache.ts provides the functionality to clear an existing Redis cache based on a predefined REDIS_KEY_PREFIX* key prefix. The API is useful to clear all cache-store entries prior to a new release/deployment.

The endpoint is automatically secured by CACHE_MANAGEMENT_SECRET.

Example:

POST /api/clear-page-cache?secret=your-strong-random-secret-key

| Query param | Description | |---|---| | secret | Must match CACHE_MANAGEMENT_SECRET. |

2. Cache populate API route

The copied api/populate-page-cache.ts is an example handler that batch-revalidates pages by fetching paths from your sitemap. It is designed to be triggered post-deployment to pre-warm the cache.

Note: This file is intended to be customised for your project's implementation. The example uses Sitecore JSS's sitemapFetcher from lib/sitemap-fetcher, a Sitecore JSS implementation used for static site generation. These paths could be loaded from other API calls or even a static list.

Example:

POST /api/populate-page-cache?secret=your-strong-random-secret-key&pathCount=100

| Query param | Description | |---|---| | secret | Must match CACHE_MANAGEMENT_SECRET. | | pathCount | Maximum number of pages to revalidate (default: 10). |

Environment variables can be added to manage batch processing to reduce impact on dependent resources during page generation.

CACHE_POPULATE_BATCH_DELAY_MS
CACHE_POPULATE_BATCH_SIZE

3. Cache initialisation modes

Set REDIS_CACHE_INIT to control how the cache is pre-populated when the Next.js server starts:

| Value | Behaviour | |---|---| | manual | (default) No automatic population. The cache warms organically as users visit pages, or you can trigger /api/populate-page-cache post-deployment, pre go-live. | | instrumentation-soft | On startup, uses registerInitialCache to populate the cache from the build artifact. Keys that already exist in Redis are not overwritten. | | instrumentation-hard | On startup, uses registerInitialCache to populate the cache from the build artifact, overwriting any existing keys. Be aware this can temporarily serve stale content while the cache is being rebuilt. |

The instrumentation-soft and instrumentation-hard modes rely on the instrumentation.ts registration that the postinstall script configures automatically.

The manual method can be used in conjunction with the clear-cache and populate-cache api routes during a deployment to populate the cache.

Key namespacing and separation from page caching

Page caching uses a key prefix to not overlap in deployments and other caching implementations.

1. Page cache keys use REDIS_KEY_PREFIX directly, for example blue:....
2. This ensures other implementations that use the redis instance (i.e. Dataweavers custom caching) do not conflict with keys. In addition it facilitates blue/green style deployments.