@aeyon-studios/payload-website-cache

Connects Payload publishing to Next.js cache invalidation, background page preparation, and a Website Updates admin screen. The supported baseline is Payload 3.80, Next.js 16.3 with Cache Components, and React 19.

Install

pnpm add --save-exact @aeyon-studios/[email protected]

The package includes compiled JavaScript, TypeScript declarations, admin styles, and hosting scripts. Consuming sites do not need its source code or build tools.

Configure the site

import { defineWebsiteCache } from '@aeyon-studios/payload-website-cache'

export const settings = defineWebsiteCache({
  collections: {
    pages: { route: '/:slug' },
    news: { route: '/news/:slug', refresh: ['/news'] },
    media: {},
  },
  globals: { navigation: { refresh: ['/'] } },
  preparePaths: ['/'],
})

route identifies a document's page. refresh lists extra URLs to refresh after publication. It can also be a function receiving { before, after, payload } and returning URLs. Query filters and card counts belong to the components using them.

import { createWebsiteCache } from '@aeyon-studios/payload-website-cache/server'
import { settings } from '@/website/cache.config'

export const websiteCache = createWebsiteCache(settings, {
  config: async () => (await import('@/payload.config')).default,
})

Add websiteCache.plugin to Payload's plugins. Wrap Next's configuration with withWebsiteCache from @aeyon-studios/payload-website-cache/next. Register websiteCache.POST and websiteCache.GET at /api/website-cache/[action].

Cached reads and streaming

The site owns its normal cached reads using Next's cache directives. The /core entry provides query and dependency tags. websiteCache.canCache lets reads fall back to fresh content when a rule or the background worker is missing. There is no query registry to maintain.

Enforce anonymous access and published-only content in public reads. Authorized draft previews must bypass the public cache. Never cache user-specific results with public tags.

A section inside Suspense can stream while its query result comes from the cache. For random cards, cache the candidate list and select items inside a request-time boundary. Streaming does not require a database query on every visit.

What publishing does

1. Saves a refresh job in the same transaction as the published content.
2. Expires affected data tags, old and new document URLs, and configured extra URLs.
3. Visits those URLs and reads their complete HTML and navigation responses.
4. Records the latest successful refresh, then removes the finished job. Failed work retries and remains visible if retries are exhausted.

Preparation is best effort. A visitor can reach an expired page before its worker. Related data is invalidated without scheduling every page that references it. Only queries used by the prepared URLs are warmed; other filters and pagination fill on demand.

Website Updates

The dashboard checks every three seconds while visible, and immediately when the user returns to its tab. Waiting and Preparing show unfinished jobs, so a quick refresh can finish between two checks and correctly leave both at zero.

A single “Last refresh completed” confirmation shows the content and completion time, even if the job finished before the dashboard opened. It is saved only after invalidation and preparation succeed, using Payload's existing key-value store. The next completion replaces it; no activity history or new collection is added. The dashboard also shows its last successful check time and keeps the previous status visible if its connection is interrupted.

Node and Docker

Set PUBLIC_CACHE_JOBS=1 and a private WEBSITE_REDIS_URL. Set PUBLIC_CACHE_PREPARE=1 to keep readiness false until preparation completes. Keep a stable WEBSITE_CACHE_NAMESPACE per site and environment, and a unique NEXT_PUBLIC_BUILD_SHA per deployment.

The package includes handlers for Next's component cache and incremental route cache. This application entry starts standalone server.js and prepares its public URLs:

// scripts/start-prepared.mjs
import '@aeyon-studios/payload-website-cache/start'

Include this script and the installed package's runtime files in the standalone image. Run node scripts/start-prepared.mjs. Set WEBSITE_SERVER_FILE if the server is elsewhere. The package checks Redis storage before and after preparation and exposes websiteReady for the site's health endpoint.

Redis should use persistent storage and a memory limit with volatile-lru. Cached page entries expire; invalidation markers remain protected. Do not use allkeys-lru. Older installations must finish migrating to permanent invalidation markers before changing policy.

The hosting platform must keep the previous instance serving until the replacement is healthy. This package does not configure proxy handover or database backups.

Vercel

Use PUBLIC_CACHE_JOBS=1 and CRON_SECRET, register the authenticated cron handler, and build with buildPreparedWebsite. Production builds generate and verify the configured page shells. Preview deployments use a normal build with publication jobs disabled. The cron drains up to ten jobs per request, stopping between jobs after thirty seconds. Set the route's maximum duration to allow the current job to finish.

Vercel uses its native cache storage; it does not require Redis. Build-time component entries are not guaranteed to be seeded into Vercel's remote cache. Streamed sections may fill their data cache on first use even when the page shell was prepared. Post-deployment preparation is a separate optional step; this package does not install a promotion workflow. Protected deployments need VERCEL_AUTOMATION_BYPASS_SECRET for worker requests. Validate deployment and promotion in each adopting site.

Plugin development

pnpm install --frozen-lockfile
pnpm check
pnpm test
CACHE_TEST_REDIS_URL=redis://127.0.0.1:56401 pnpm test
pnpm pack

The Redis tests require a disposable loopback Redis instance. Some tests change its memory limit to test eviction, so never use a project cache or production Redis. Never point application fixtures at a live CMS database.

Install the generated tarball into a consuming site to test the actual packaged files before publishing. Bump the package version for each release, then update each site's pinned dependency when ready. A plugin release does not update existing sites automatically.

Shared build and redirect helpers

Keep site URLs in cache.config.ts and connect Payload in cache.ts. A site with derived catalog URLs can keep those queries in a third file. The package owns inventory, build checks, generated manifests, redirect caching and migration adapters.

preparePaths still accepts an array. For derived URLs it can also be an async function receiving the Payload instance. This function runs during deployment preparation, not on each publish. Collection refresh rules continue to decide which pages a publication prepares.

buildPreparedWebsite(websiteCache) prepares and verifies every public route for Vercel. buildContainer(websiteCache) reads anonymous public URL samples, builds without database access and includes the startup package with its dependencies. Container startup then prepares the full inventory. Import these build helpers from @aeyon-studios/payload-website-cache/build. Both write .website-cache/prepared.json, which should be ignored by Git. withWebsiteCache includes that generated file in page deployments and bundles its public redirects into the proxy. A cold proxy can apply those redirects without reading a runtime file. It refreshes its snapshot in the background.

Each native Next page supplies its build parameters with the same helper:

import { createStaticParams } from '@aeyon-studios/payload-website-cache/deployment'

export const generateStaticParams = createStaticParams('/news/[slug]')

Use the page's URL pattern, without route-group folders such as (frontend). The helper reads the prepared public URLs and Next's compiled route map. Static pages and specific dynamic routes take precedence over catch-alls automatically. Adding /exhibitions/[slug] does not require changing another page's exclusions. Derived listings in preparePaths follow the same rules.

Adding a content type means defining its public URL in cache.config.ts and adding its native Next page with this helper. No additional route registry or collection filter is needed. Collections without public pages do not need a page or this helper. An omitted URL rule cannot be prepared ahead of time; normal Next rendering still handles the page.

The helper returns the build function, or undefined when there are no examples. This avoids Next’s Cache Components error for empty generateStaticParams results without inventing placeholder URLs. Development and Vercel previews also omit this optional build hook. Next writes its route map before evaluating pages for build preparation. The helper reads it only during that build phase; visitor requests never read these build files. A missing map or a misspelled pattern fails preparation with a specific error instead of assigning URLs to the wrong route. Rendering, metadata, ISR invalidation and background warming remain unchanged.

The package uses Next's internal route-map format and matching helpers in this build-only integration. These are not a public Next API; test the integration against a real Next build when upgrading the supported Next version. No Next routing internals are used to dispatch visitor requests.

Import createRedirectProxy from @aeyon-studios/payload-website-cache/proxy to share redirect refresh and fallback behavior. Its default endpoint is /api/cache/redirects; pass endpoint to use another URL. The endpoint can resolve public redirect documents with websiteCache.resolveRedirects(docs) after the site's normal cached query.

Use custom: { websiteCache: false } on private collections. They are excluded from hooks, inventory and cache eligibility. allowMissing on a route rule marks documents that intentionally return 404, such as unavailable catalog items; other missing pages still fail build verification.

websiteCache.legacyTasks(options) is optional support for a previous job queue. The project supplies its old task names, queue names, cache tags and a mapChange function that translates its old records. The package does not assume project field names or install these tasks automatically. New projects do not need this. Remove the project configuration once its old queues and rollback window are finished.

Shared releases must be tested with both NYDC and 33sixty before upgrading either project. Keep content rules in each project; a package release must not assume products, showrooms, news or any particular collection exists.

Integration requirements

• Public reads must return only published content and enforce anonymous access. Use custom: { websiteCache: false } for submissions, internal records and other private collections.
• The built-in lookup tags understand id, slug, pathname and pageType. For other unique lookups, the collection tag safely invalidates the cached query. Fixed ID sets use document tags, including IDs missing from the current result.
• Pass query options to cachedReferences(payload, fields, data, options) so depth-zero ID-only reads do not depend on unrelated document content. Populated relationships and joins still receive dependency tags.
• Inventory reads route fields such as slug, pathname and path. Content labels use title, then name. Put custom URL rules in the site configuration.
• Container builds need publicly readable REST collections at /api/<collection> on CMS_BUILD_URL. Authentication and private collections are excluded.
• Redirect helpers use the redirects collection and Payload's redirects-plugin document shape. Sites provide the read endpoint when using a different path.
• Mount package handlers at /api/website-cache/[action]. Container /api/health must return sha: buildRevision and the package's readiness value. Use the same build revision throughout startup and health checks.
• Give each environment its own WEBSITE_CACHE_SECRET, Redis namespace and appropriate origin. Environments sharing a Payload secret otherwise share the derived worker token.
• Job concurrency control is always enabled. Generate and apply the Payload database migration before enabling workers; environment flags must not change the schema.

Page-cache Redis reads fail open during an outage, then reconnect without restarting the application. Failed invalidation rejects the publication job so it can retry. This is different from serving stale content and reporting a successful refresh.

The optional legacy adapter runs old warm jobs in a Next.js request context. Do not drain those jobs from a standalone Payload worker. Remove the adapter after old queues and the rollback window are finished.