@purposeinplay/payload-image-translate

Localize text-bearing image assets (banners, promotions, tournament art) for Payload CMS 3 — regenerate an image per locale with the baked-in text translated and everything else preserved. The visual analog of payload-ai-translate.

Generation never touches the live field directly: every render becomes a review candidate that an editor approves before it goes live.

How it works

A generative image-edit engine with an OCR safety net:

source media (en)
  ├─ extract    Claude vision reads the text runs + per-run style + bounding box
  ├─ translate  exact target strings (built-in Claude translator, or reuse ai-translate)
  ├─ frame      REGION: regenerate only a window around the text, composite back
  │             (text position locked; everything else stays byte-identical) —
  │             or full-frame pad/crop-back when no usable text box
  ├─ render     gpt-image-1.5 images.edit (input_fidelity: high), exact-string prompt
  ├─ verify     Claude vision checks spelling + edge-clipping → retry (≤3)
  └─ candidate  staged for editor review; on approve, written to the localized field

The model is never asked to translate — it renders the exact pre-translated string, and the result is OCR-verified, so the misspellings that generative image models routinely produce are caught and retried. Ultra-wide banners (4:1+) go through the region-edit path, which anchors the text at its original position and never regenerates the rest of the artwork.

Quick start

import { imageTranslatePlugin } from '@purposeinplay/payload-image-translate';
import {
  createAnthropicVisionProviders,
  createOpenAIImageProvider,
} from '@purposeinplay/payload-image-translate/providers';

const vision = createAnthropicVisionProviders({ apiKey: process.env.ANTHROPIC_API_KEY! });

export default buildConfig({
  plugins: [
    imageTranslatePlugin({
      collections: {
        // FIELD mode — a named localized `upload` field on a content collection
        'promotions-v2': { fields: ['featured_image'] }, // must be `localized: true`
        'banners-v2': { fields: ['image'] },
        // UPLOAD mode — the media doc's OWN file, with its alt text alongside
        media: { upload: true, altField: 'alt' },
      },
      sourceLocale: 'en',
      targetLocales: ['de', 'es', 'fr', 'it', 'pt', 'ru', 'tr', 'ja', 'ko', 'zh'],
      mediaCollectionSlug: 'media',
      costPerRenderUsd: 0.1, // budget meter; baseline against your real bill
      engine: {
        extractor: vision.extractor,
        translator: vision.translator,
        verifier: vision.verifier,
        imageEditor: createOpenAIImageProvider({ apiKey: process.env.OPENAI_API_KEY! }),
      },
    }),
  ],
});

Then, in the consumer: make each target upload field localized: true, run payload generate:importmap (registers the editor UI), and create + run a DB migration (the plugin adds a reviews collection). Full walkthrough in docs/getting-started.md.

Upgrading to 0.5.1 needs a migration. It adds errorClass (select) and errorDetail (json) to the reviews collection and a queued value to its status enum. Nothing is dropped. See docs/upgrading.md.

Forwarding failures to your error tracker

Every terminal failure — a failed render, a cost-guard refusal, a job that couldn't be queued, a queued row the crash sweep gave up on — is classified once and handed to an optional onError hook, so it reaches Sentry with the cause already named instead of being reconstructed from a log line:

imageTranslatePlugin({
  // …
  onError: (err, ctx) => {
    Sentry.captureException(err, {
      tags: { plugin: 'image-translate', phase: ctx.phase, errorClass: ctx.errorClass },
      extra: { ...ctx.detail, doc: `${ctx.collection}/${ctx.docId}`, locale: ctx.locale },
    });
  },
});

ctx.errorClass is one of transient · auth · quota · moderation · input · cap · cancelled · timeout · unknown — the same value persisted on the review row and the same one that decides whether the editor is offered a Retry button.

Two modes

| Mode | Subject | Config | |---|---|---| | Field | A named localized upload field on a content collection (featured_image) | { fields: ['featured_image'] } | | Upload | The collection's own uploaded file — the media doc itself, where the file is the document | { upload: true, altField: 'alt' } |

They're mutually exclusive per collection; supplying both throws at startup. Upload mode is how you localize a media library in place, and it adds a status column to that collection's list view.

Editors localize from a "Localize image" button next to the image field: pick languages → watch a progress board → review and approve per locale.

There is also a freestanding Image Translation Studio admin view for ad-hoc assets that don't live on a document: upload or pick any Media image, generate per-locale variants, compare against the source with a wipe slider, download, and optionally save to the Media library (nothing is persisted until you save). See docs/studio.md.

Operators get runtime kill switches (an auto-registered image-translate-settings global): turn off the document flow and/or the Studio from the admin in seconds, no deploy — new generation stops, already-paid renders stay reviewable.

Endpoint access — configure it

The plugin's endpoints are reachable by any principal Payload authenticates, and Payload authenticates more than admin-panel sessions: it registers an API-key auth strategy for every collection with auth.useAPIKey and runs all of them on every request, custom endpoints included. A machine key from an unrelated collection is a valid req.user with no roles.

So the gate fails closed: only principals from the admin panel's user collection (config.admin.user) are accepted, and with no callback configured, nothing weaker. A consumer callback can narrow that further — never widen it.

imageTranslatePlugin({
  access: {
    review: (user) => Boolean(user?.roles?.some((r) => r === 'admin' || r === 'editor')),
    bulk:   (user) => Boolean(user?.roles?.includes('admin')),
    strict: true,
  },
  // …
})

access.strict controls what happens to a principal that fails only the structural check. It defaults to false in this release: the request is allowed through and a logger.warn names the principal and the route, so you can find your gaps before anything breaks. The default flips to true in the next patch release — set your callbacks now, watch the logs until they are clean, then set strict: true yourself rather than waiting for the flip.

Denials that were already 403 stay 403 regardless of strict: no session, and an explicit access.review / access.bulk callback that returns false.

Details, the full endpoint→access table and the rest of the security model: docs/security-and-cost.md.

Running a dedicated jobs worker

Renders run as Payload jobs on the default queue. After queueing, /generate and /retry kick the job runner so work starts without waiting for the cron tick — and that kick executes the jobs in the calling process, i.e. the web container. If you run a separate worker, turn it off:

imageTranslatePlugin({
  // wild/wolf/raptor run `pnpm jobs:worker` (JOBS_DEDICATED_WORKER=true), so
  // the web tier must not execute renders: a 13-locale generate would start
  // up to 13 multi-minute image renders inside the request container.
  kickQueue: false,
  // …
});

With kickQueue: false nothing is run in-process; the worker's cron claims the jobs on its next tick (one minute, in wild's config). Leave it at the default true when Payload's job runner is only ever driven from the app itself.

When the kick is enabled it is now scoped: where: { taskSlug } so it can only claim image renders (never a queued bulk-translate-doc), and sequential: true so the locales run one at a time rather than all at once.

Documentation

| Doc | What's in it | |-----|--------------| | Getting started | Install, env, providers, the consumer setup steps | | Configuration | Complete ImageTranslatePluginConfig reference | | Editor guide | How editors use the button → progress → review/approve flow | | Studio | The freestanding Image Translation Studio view (upload → generate → compare → save) | | Engine | Pipeline internals, region-edit vs full-frame, providers, OCR verify-retry, headless usage | | API endpoints | The REST endpoints under /api/image-translate/* | | Architecture | Review-candidate state machine + document lifecycle | | Security & cost | SSRF, access/IDOR, input caps, cost guards, kill switches | | Operations | Runbooks: stuck renders, the janitor, crash recovery, cost guards | | Upgrading | Per-version migrations and behaviour changes | | Diagrams | The 31-act Excalidraw lifecycle map | | Roadmap: rich-text image localization | Spec for the deferred rich-text version |

Requirements

Peer dependencies: payload ^3, @payloadcms/ui ^3, react 18/19, zod. Runtime: @anthropic-ai/sdk, openai, sharp. Needs ANTHROPIC_API_KEY and OPENAI_API_KEY (or your own providers — see engine).

OpenRouter is supported as an alternative route for both roles — createOpenRouterVisionProviders and createOpenRouterImageProvider — which lets you reach the same models (or others) through one key. Register them in providerCatalog and admins can switch at runtime from the settings global, without a deploy.

Publish gotcha: main/exports point at src/*.ts for in-repo dev; the real dist/*.js paths live in publishConfig. Publish with pnpm publish, not npm publish — npm does not apply publishConfig field-overrides.

Headless (no Payload)

import { translateImage } from '@purposeinplay/payload-image-translate';
const { renders } = await translateImage(sourceImage, targetLocales, engine);

Status & limitations

Validated on real assets across all 10 locales (incl. CJK/Cyrillic/Turkish). Works on localized upload fields (field mode) and on upload collections such as a media library (upload mode). Images embedded inside rich text are not yet supported — that's a deferred, separately-architected version (see the roadmap spec). Other known areas: scene drift on brand-critical elements (use the approval gate), content-policy refusals on some imagery, and costPerRenderUsd is an operator estimate — baseline it against your real OpenAI bill. See security & cost.