Medusa Payload Sync

This Medusa shop component mirrors the small set of Medusa product and variant facts needed by Payload projects. Medusa remains the source of truth for catalog and commerce data. Payload remains the source of truth for editorial content.

The plugin writes only two hidden, protected mirror collections. Editorial documents live in separate, project-defined collections, so publishing a Medusa title or handle can never publish an editorial draft.

Support

• Medusa: ^2.19.0
• Payload: 3.x REST API
• Node.js: >=20
• Package manager: pnpm ([email protected] in this workspace)

All Medusa 2.x peer dependencies accept ^2.19.0; development dependencies remain pinned to 2.19.0.

Installation in a shop

pnpm add @hartl-services/medusa-payload-sync @hartl-services/medusa-base

@hartl-services/medusa-base (>= 1.0.0) is a required peer: it stores the plugin switch (see Feature toggles), the connection settings (see Settings in the Admin) and runs the sync_now action. Register it as a plugin as well, before Payload Sync.

// medusa-config.ts
const plugins = [
  { resolve: "@hartl-services/medusa-base", options: {} },
  {
    resolve: "@hartl-services/medusa-payload-sync",
    // No connection options: URL, API key and auth collection are Admin
    // settings. Only the config-only options below may be passed here.
    options: {},
  },
  // ...other plugins
];

No option is required. The Payload URL, API key and auth collection are runtime settings maintained only in the Admin (see Settings in the Admin); there is no medusa-config.ts or environment fallback. The former options payloadUrl, payloadApiKey and payloadAuthCollection are rejected at boot with an error that points to the Admin settings. The config-only options (collections.product, collections.variant, requestTimeoutMs, backfillBatchSize, backfillBatchDelayMs) are described in Options and exact defaults below. Also complete Configure Payload authentication below, which the plugin itself does not automate.

Settings in the Admin

The plugin declares three settings to @hartl-services/medusa-base. An Admin maintains them under Settings → "Hartl Services Plugins" → Payload-CMS-Sync ("Öffnen"):

| Key | Label | Type | Required | Default | | ------------------------- | --------------- | ------ | -------- | ------- | | payload_url | Payload-URL | url | yes | none | | payload_api_key | API-Key | secret | yes | none | | payload_auth_collection | Auth-Collection | string | no | users |

Precedence per key: value stored in the Admin > default. The page shows each value's source; the API key is stored encrypted by the base plugin and never displayed ("gesetzt" only). "Löschen" removes a stored value: the URL or API key is then missing (the plugin is "nicht konfiguriert"), the auth collection returns to users. Saving a changed URL or auth collection also clears the stored API key (the settings declare invalidates: ["payload_api_key"]), so the key is never sent to a host it was not entered for; the plugin is "nicht konfiguriert" until the key is entered again. Rotating the backend's COOKIE_SECRET makes a stored API key unreadable; it then counts as not set until it is saved again (see the base plugin's README).

Every sync resolves the settings anew, so a change applies to the next event without a restart. The settings are read at app level, never by the module service (a module does not reach into other modules):

• resolvePayloadConnection(container) (src/lib/payload-connection.ts) reads them through the base plugin with the caller's application container (workflow step, subscriber, script), validates them and returns the connection; while incomplete it throws the permanent PayloadSyncNotConfiguredError.
• describePayloadSyncConfiguration(container) answers the same check without throwing (subscriber guard, reconciliation script, sync_now).
• Every Payload-facing method of the module service takes that connection as its first parameter, for example service.syncProductSnapshot(connection, snapshot) or service.findProductMirror(connection, medusaId). The service builds its REST client from it lazily and reuses it while URL, key and auth collection stay the same. An injected payloadSyncRestClient (test seam) is used as-is and the connection is ignored. Resolved values are validated like the former options (http/https URL with one trailing slash removed, non-empty key, lowercase auth-collection slug); an invalid value counts as missing. The collections, timeout and backfill options stay config-only.

Not configured

While URL or API key is missing or invalid, the plugin is "nicht konfiguriert". That is also the state of every fresh installation until the settings are saved:

• the backend boots normally and does not contact Payload at startup;
• the settings page and the plugin hub show "nicht konfiguriert" with the missing keys;
• the subscribers check the plugin switch first, then the settings, and return without mirroring; product and variant changes made meanwhile are not caught up automatically (run the reconciliation afterwards);
• the reconciliation script returns an empty summary without touching Payload; the sync_now action ends failed with the not-configured message;
• one warning per process names the missing or invalid keys (never values), for example Payload-CMS-Sync ist nicht konfiguriert (fehlend: payload_url).; after a successful resolution the warning is re-armed for the next outage.

Migrating an existing shop

Earlier versions read the connection from medusa-config.ts (PAYLOAD_URL, PAYLOAD_SERVICE_API_KEY, PAYLOAD_AUTH_COLLECTION). The settings only exist from this version on, so upgrade in this order:

1. Remove the three options from the plugin entry (the boot fails while they are still passed) and drop the environment variables.
2. Deploy. The plugin reports "nicht konfiguriert"; product and variant events in the meantime are not mirrored.
3. Enter URL, API key and auth collection in the Admin (Settings → "Hartl Services Plugins" → Payload-CMS-Sync).
4. Run "Jetzt synchronisieren" to catch up on the changes missed meanwhile.

Unlike the Documents module of @hartl-services/medusa-base, which deliberately keeps its merchant option as a config fallback for the shop master data, this plugin has no config or environment fallback for its connection settings.

Admin action "Jetzt synchronisieren"

The descriptor declares the action sync_now ("Jetzt synchronisieren", mode background, with a confirmation prompt). It runs the workflow payload-sync-reconcile-catalog, which calls the same full reconciliation as the script. The base answers 202 right after starting it and emits plugin-action.finished.v1 with completed or failed when it ends; any failed product or a failed orphan scan makes the run failed. The base records every run and refuses a second start while one is running (409).

A failed run's message names the counts and the classified cause, redacted, e.g. Payload-Abgleich fehlgeschlagen: 4 von 4 Produkt(en) nicht gespiegelt (Payload nicht erreichbar, Netzwerkfehler [PAYLOAD_NETWORK_ERROR]); verwaiste Spiegel nicht geprüft (…). A retryable error (network, timeout, HTTP 408/429/5xx) makes Medusa schedule the step's retry (retryInterval) instead of retrying inline; the reconciliation does not wait for it and counts that product or orphan page as failed right away. Rerun once Payload is back.

The workflow reports its progress to that run through the base's reportPluginActionProgress after every page, and the Admin shows it while the run is active:

• product pages: current = products scanned, total = products in Medusa, label e.g. Produkte: Seite 3, 140 gespiegelt;
• orphan scans: Verwaiste Produkt-Spiegel: Seite 1, Verwaiste Varianten-Spiegel: Seite 1;
• at the end: Fertig: 140 Produkte und 312 Varianten gespiegelt, 2 verwaiste Spiegel auf Entwurf gesetzt (plus , N fehlgeschlagen for failed products).

The reconciliation script reports nothing (it has no run). While the plugin is "nicht konfiguriert" the action ends failed with the not-configured message (for example Payload-CMS-Sync ist nicht konfiguriert (fehlend: payload_url).) and no progress.

Feature toggles

The plugin declares a plugin switch without features to @hartl-services/medusa-base under the plugin key payload_sync. An Admin switches it under Settings → "Hartl Services Plugins"; its effective value is readable from GET /store/plugin-features/payload_sync. The plugin has no HTTP routes.

| Key | Label | Default | Gates | Keeps running when off | | --------------- | ---------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | | (plugin switch) | Payload-CMS-Sync | on | subscribers payload-sync-product-upserted, payload-sync-product-deleted, payload-sync-variant-upserted, payload-sync-variant-deleted | the module, existing Payload mirror documents (unchanged), the manual reconciliation script and the sync_now action |

While the switch is off, product and variant events are consumed without effect: nothing is mirrored and nothing is caught up when the switch is turned on again. Run the reconciliation afterwards to bring Payload back in line; neither the script nor the sync_now action reads the switch. A toggle or settings read failure fails the subscriber, so the Event Bus retries it.

Configure Medusa

This repository's demo/test host, apps/backend, configures Payload Sync as a local workspace dependency directly in apps/backend/medusa-config.ts, following the same direct medusa-config.ts entry shown above; it does not ship a Payload runtime package.

Feature selection and non-secret defaults belong directly in the plugin's options in apps/backend/medusa-config.ts. It is installed by being listed there; the Admin plugin switch (see Feature toggles) only pauses the mirroring.

The demo shop passes no Payload connection: it boots with the plugin "nicht konfiguriert" until URL and API key are saved in the Admin. Never forward the API key to a Storefront or Admin client.

The backend configuration owns these server-only options directly; Payload Sync exports no consumer-facing option or HTTP contract.

Options and exact defaults

| Option | Required | Implemented default | Purpose | | ----------------------- | -------- | ------------------------- | ---------------------------------------------------------------------------------------------- | | collections.product | no | medusa-products | Product mirror collection slug. | | collections.variant | no | medusa-product-variants | Variant mirror collection slug. | | requestTimeoutMs | no | 10000 | Timeout for one Payload REST request; accepted range 100–120000. | | backfillBatchSize | no | 50 | Medusa/Payload page size for reconciliation; accepted range 1–500. | | backfillBatchDelayMs | no | 250 | Delay between non-final Medusa product pages; accepted range 0–60000. |

Collection slugs must be lowercase Payload-style slugs. Invalid options and the removed connection options fail the boot. The connection settings are validated when they are used (an invalid one makes the plugin "nicht konfiguriert"). apps/backend/medusa-config.ts passes no options; changes to non-secret shop defaults must be modeled directly in the plugin's options there.

The REST client rejects any successful JSON response larger than 8 MiB with PAYLOAD_RESPONSE_TOO_LARGE and cancels the response stream. This fixed bound keeps a malformed or unexpectedly expanded Payload response from consuming unbounded memory while leaving ample room for /api/access and the maximum configured page of 500 deliberately minimal mirror documents. Keep editorial and other large fields out of the mirror collections rather than relying on the limit as storage capacity.

Configure Payload authentication

Create a dedicated user in one Payload auth collection and enable Payload's standard API-key strategy on that collection. For the default auth slug:

import type { CollectionConfig } from "payload";

export const Users: CollectionConfig = {
  slug: "users",
  auth: {
    useAPIKey: true,
  },
  fields: [
    // project-specific user fields
  ],
};

After restarting Payload with useAPIKey: true:

1. Create one account dedicated to this integration in the configured auth collection through Payload's normal Admin or Local API.
2. Enable API-key authentication for that account and copy the generated key once into the Medusa server's secret store.
3. Copy the account's Payload document ID into the Payload server environment.

Keep the resulting values server-side:

• its document ID as PAYLOAD_SYNC_SERVICE_USER_ID in the Payload process;
• its API key in the Admin setting "API-Key" (payload_api_key).

Set the Admin setting "Auth-Collection" (payload_auth_collection) to the auth collection's actual slug. The plugin sends the standard header Authorization: <auth collection> API-Key <API key> on every Payload request. It never logs this header or the key. Do not put either secret in a public environment variable, client bundle, URL, or checked-in .env file.

No custom Payload endpoint, hook, field factory, or companion package is required.

Payload mirror collection contract

Both mirror collections enable Payload drafts, stay hidden from the main Admin navigation, deny every delete, and give only the dedicated service user create and update access. Any authenticated Payload user may read them. Anonymous reads receive only published documents through an explicit _status filter.

Place the shared helpers in a server-only Payload source file:

// payload-sync-access.ts
import type { Access, FieldAccess } from "payload";

export const isPayloadSyncService: Access = ({ req }) =>
  Boolean(
    req.user?.id &&
    String(req.user.id) === process.env.PAYLOAD_SYNC_SERVICE_USER_ID,
  );

export const syncFieldUpdate: FieldAccess = ({ req }) =>
  Boolean(
    req.user?.id &&
    String(req.user.id) === process.env.PAYLOAD_SYNC_SERVICE_USER_ID,
  );

PAYLOAD_SYNC_SERVICE_USER_ID must be set before Payload starts. A missing or incorrect ID denies mirror writes.

Product mirror

// collections/MedusaProducts.ts
import type { CollectionConfig } from "payload";

import { isPayloadSyncService, syncFieldUpdate } from "../payload-sync-access";

export const MedusaProducts: CollectionConfig = {
  slug: "medusa-products",
  admin: { hidden: true, useAsTitle: "title" },
  versions: { drafts: true },
  access: {
    read: ({ req }) => (req.user ? true : { _status: { equals: "published" } }),
    create: isPayloadSyncService,
    update: isPayloadSyncService,
    delete: () => false,
  },
  fields: [
    {
      name: "medusaId",
      type: "text",
      required: true,
      unique: true,
      index: true,
      admin: { readOnly: true },
      access: { update: syncFieldUpdate },
    },
    {
      name: "title",
      type: "text",
      required: true,
      admin: { readOnly: true },
      access: { update: syncFieldUpdate },
    },
    {
      name: "handle",
      type: "text",
      required: true,
      index: true,
      admin: { readOnly: true },
      access: { update: syncFieldUpdate },
    },
  ],
};

medusaId is the only identity key and must stay unique. handle is intentionally not unique. If a deleted Medusa product is later replaced by a new product using the old handle, its new medusaId identifies a new mirror document. A title or handle change with the same medusaId updates the existing document and Payload records the revision.

Field-level access does not replace collection access: the service user still needs collection-level update permission for field updates to succeed.

Variant mirror

// collections/MedusaProductVariants.ts
import type { CollectionConfig } from "payload";

import { isPayloadSyncService, syncFieldUpdate } from "../payload-sync-access";

export const MedusaProductVariants: CollectionConfig = {
  slug: "medusa-product-variants",
  admin: { hidden: true, useAsTitle: "title" },
  versions: { drafts: true },
  access: {
    read: ({ req }) => (req.user ? true : { _status: { equals: "published" } }),
    create: isPayloadSyncService,
    update: isPayloadSyncService,
    delete: () => false,
  },
  fields: [
    {
      name: "medusaId",
      type: "text",
      required: true,
      unique: true,
      index: true,
      admin: { readOnly: true },
      access: { update: syncFieldUpdate },
    },
    {
      name: "product",
      type: "relationship",
      relationTo: "medusa-products",
      required: true,
      index: true,
      admin: { readOnly: true },
      access: { update: syncFieldUpdate },
    },
    {
      name: "title",
      type: "text",
      required: true,
      admin: { readOnly: true },
      access: { update: syncFieldUpdate },
    },
  ],
};

When collections.product or collections.variant uses a custom slug, change the corresponding Payload slug values and change the variant relationTo to the configured product mirror slug as well.

Register Users, MedusaProducts, and MedusaProductVariants in the project's normal payload.config.ts. The plugin does not create or migrate these collections.

Editorial isolation

Mirror documents contain only the plugin-owned facts shown above. Do not add editorial Rich Text, SEO, media, compliance, or other mutable content fields to them. Create any editorial collections your project needs and relate them to the product or variant mirror using normal Payload relationships.

The plugin neither requires nor inspects those editorial collection names, fields, access rules, relationships, drafts, or publishing workflow. It never reads or writes editorial documents. This is the key editorial safety property: Medusa immediately updates and publishes only hidden mirror facts, so it cannot publish or overwrite an in-progress editorial draft.

State and publishing invariant

Every mirror write sends _status: "draft" or _status: "published" explicitly. The plugin does not use draft=true as a status transition and never hard-deletes a Payload document.

| Medusa state/action | Product mirror | Current variant mirrors | Historical/deleted variant mirrors | Editorial documents | | ----------------------- | --------------------- | ----------------------------- | ---------------------------------- | ------------------- | | Published create/update | published immediately | published immediately | draft | untouched | | draft/proposed/rejected | draft immediately | draft immediately | draft | untouched | | Product delete | draft | draft | draft | untouched | | Product republish | published immediately | published immediately | draft | untouched | | Variant delete | unchanged | deleted variant becomes draft | draft | untouched |

On every product sync, the plugin queries the current Medusa variants. A published product publishes exactly those current variants and drafts mirror variants no longer present in Medusa. Republish therefore restores all current variants but never resurrects historical or deleted variants. A draft, proposed, rejected, or deleted product drafts its product mirror and all related variant mirrors immediately.

A standalone variant sync first ensures the parent product mirror exists and uses its Payload document ID for the required relationship. The variant follows the current Medusa parent product status.

Repeated events are idempotent by medusaId. Payload's required unique index is the concurrency guard for parallel creates. Payload 3.87 reports that race as a structured HTTP 400 ValidationError; the client recognizes only the exact configured-collection medusaId shape, reads the winner once, and updates it with the requesting snapshot. Legacy HTTP 409 conflicts remain compatible. Other validation errors remain ordinary permanent 4xx failures. Safe repeat processing may create additional Payload versions, but not additional mirror identities.

Ordering and locking contract

Product sync, variant sync, product deletion, variant deletion, and both orphan reconciliation page types share one coarse payload-sync:catalog lock. After acquiring it, the workflow reads the authoritative Medusa entity again and then performs the required Payload writes before releasing the lock. A delayed old event therefore converges to the current Medusa state instead of winning merely because its Payload request completed last.

The plugin uses Medusa 2.19.0's public owner-aware acquire and release operations rather than locking.execute. Every attempt has a random owner ID, a 15-second acquisition deadline, and a 30-second expiring lease. While the authoritative callback is active, the same owner renews the lease every 10 seconds. Lease continuity is tracked conservatively from the start of each successful provider call. A heartbeat that starts at or after the last confirmed deadline is not sent, and a renewal first observed at or after that deadline is not accepted as continuity even if the provider reports success.

Individual acquire, renew, and release provider calls are locally observed for at most their configured deadline. A timed-out provider promise cannot be cancelled; if it later acquires or renews a lease, cleanup uses only that attempt's original owner ID and is itself locally observed for a finite period. An unresolved cleanup can therefore remain in flight, but it cannot delete a successor owner's lease and the provider-side lease remains finite. The heartbeat timer is cancelled before the normal owner-specific release, including error paths. A stopped process therefore leaves a finite lease rather than an ownerless permanent lock.

If renewal or release ownership is lost while a callback is already active, the callback is allowed to settle, its stale owner cannot release a successor's lease, and the plugin acquires a fresh owner for one more complete authoritative read-and-write pass. The original retryable lifecycle error is still propagated to the workflow retry boundary even when that recovery pass encounters a permanent Payload error. This means an effect that ran after lease loss is not accepted as the final completer; operators can still observe a transient state until the recovery/retry pass finishes. Lock lifecycle failures use redacted structured error codes and are never treated as permanent Payload contract failures.

Medusa's default in-memory locking provider coordinates only one backend process. A deployment with multiple workers or backend replicas must configure a shared provider that implements Medusa's owner-ID checks, same-owner lease renewal, expiry takeover, and owner-specific release. Medusa 2.19.0's Redis locking provider has those semantics when every process uses the same Redis instance. The installed Medusa 2.19.0 PostgreSQL locking provider does not let a different owner take over its expired owner row through acquire, so it does not satisfy this plugin's process-death recovery contract. Reconciliation must use the same compatible provider and lock domain as event workers. Otherwise the ordering guarantee is only process-local or does not apply.

Capability check

The plugin does not contact Payload at startup: the connection is a runtime setting, so there is nothing to check at boot. A wrong URL, key or missing permission surfaces on the first sync (structured, redacted error log) or as a failed sync_now run.

runPayloadPreflight (used by the Payload REST contract suite against a real Payload) calls Payload's standard authenticated /api/access endpoint and validates:

• reachable authentication;
• both configured collection slugs;
• collection-level read, create, and update capabilities;
• required field capabilities only when Payload exposes those field shapes.

This is intentionally a reduced check. With no custom Payload code, the standard endpoint cannot prove every field type, unique/index rule, relationship relationTo, drafts/version setting, delete ACL, or omitted field shape. The copyable schema above is the authoritative configuration contract; runtime writes and the real Payload contract test are the authoritative protocol gate.

Initial and repeat reconciliation

Reconciliation is the initial backfill and the repair tool after missed or permanently failed events. Run the package's built script; do not invoke a raw TypeScript source path when the same plugin is enabled in the backend, because that can register the workflows twice.

From the shop's root, run:

pnpm exec medusa exec \
  node_modules/@hartl-services/medusa-payload-sync/.medusa/server/src/scripts/reconcile-payload-catalog.js

The command uses the shop's configured database and resolves the Payload connection exactly like the subscribers (Admin settings > default). While the plugin is "nicht konfiguriert" it logs that once and exits without touching Payload (the Admin action fails instead). The Admin action sync_now runs the same function as the workflow payload-sync-reconcile-catalog.

Every run:

1. scans Medusa products in ascending, bounded pages;
2. invokes the normal product workflow sequentially, including current variants;
3. waits backfillBatchDelayMs only between non-final product pages;
4. scans draft-inclusive product and variant mirrors in stable medusaId order;
5. drafts published mirror orphans whose medusaId no longer exists in Medusa;
6. logs one machine-readable summary.

The summary fields are scannedProducts, syncedProducts, syncedVariants, draftedOrphanProducts, draftedOrphanVariants, and failedProductIds. Individual product failures do not stop later products or orphan scans. After all work is attempted, any non-empty failedProductIds makes the command exit with an error. Automation must require an empty list.

If a multi-ID orphan page partially succeeds and then retries, that run's drafted-orphan count can undercount mirrors that already converged on the first attempt. Mirror state and repeat-run convergence are authoritative; this is a metrics-only limitation.

The runner is restartable and idempotent: safe repeat runs converge on the current Medusa state without duplicate mirror documents. Pagination is not a database snapshot, so rerun after concurrent catalog changes. V1 deliberately has no automatic schedule; operators decide when to rerun it (script or sync_now).

Retries, logs, and incident recovery

External workflow steps use Medusa retries (maxRetries: 3, retryInterval: 2) and idempotent writes. Transient network failures, timeouts, HTTP 408, 429, and 5xx responses are retryable. Auth, validation, missing capability, duplicate-identity, invalid-response, and other ordinary 4xx contract errors are permanent, except for the handled create unique conflict. Failed event processing is propagated rather than silently acknowledged.

Structured JSON logs include the operation and classified code, plus the entity type, medusaId, event name, workflow/run identifiers, collection, HTTP status, retryability, and attempt where available. Authorization values, API-key values, and response headers are redacted. Do not add secrets to log context or paste unredacted request data into incident records.

V1 has no plugin-owned dead-letter table. Recover an incident in this order:

1. Inspect structured logs by medusaId and error code.
2. Repair the authentication, collection schema/ACL, or network cause.
3. Rerun the full reconciliation command.
4. Require failedProductIds: [] and confirm the expected draft/published product and variant states.

Because reconciliation also drafts orphans, it repairs both missing updates and missed delete events without deleting Payload history.

V1 out of scope

• Payload editorial/content collections, Rich Text, media, SEO, or compliance schemas
• a Payload package, hook, custom endpoint, field factory, or automatic schema creation
• Storefront integration or merging Medusa and Payload responses
• Medusa categories, collections, prices, inventory, Sales Channels, or other commerce facts
• Medusa Module Links or defineLink
• a plugin-owned Admin UI or HTTP route (settings and the sync_now action are served by @hartl-services/medusa-base), or an automatic schedule
• Payload hard delete or a configurable delete mode
• a plugin-owned dead-letter database/table

Projects are free to add any separate Payload collections and relationships needed for their editorial model; those remain outside this plugin's contract.

Plugin development contract

Payload Sync is server-to-server integration code and intentionally exposes no Store or Admin HTTP routes of its own; its settings and action go through the base plugin's /admin/plugin-features routes. It has no public DTO, schema, path, Storefront client, or runtime Payload-credential contract.

If this component gains a Medusa HTTP route, define its request/query/response DTOs with @medusajs/framework/zod, derive public types from those schemas, add them and a small path object to @hartl-services/medusa-food-supplements-contracts, and let consumers use their own Medusa SDK with sdk.client.fetch<T>.