@assemblyltd/payload-chunked-upload

Payload CMS 3.x plugin that splits large file uploads into chunks on the client and reassembles them server-side. Bypasses Cloudflare's 100 MB per-request body limit.

Why

Cloudflare's free / Pro plans cap a single HTTP request body at 100 MB. Direct uploads of large files (sales brochures, hi-res images, mp4s) through a Payload admin sitting behind Cloudflare get rejected at the edge before the request reaches the origin.

This plugin makes the editor's file picker split files into ≤25 MB chunks, POST each separately (each one well below the cap), and reassemble them server-side before handing the buffer to Payload's normal upload pipeline (Sharp resize, imageSizes, write to staticDir, all hooks).

Install

pnpm add @assemblyltd/payload-chunked-upload

Usage

import { buildConfig } from 'payload'
import { chunkedUploadPlugin } from '@assemblyltd/payload-chunked-upload'

export default buildConfig({
  collections: [Media /* must have type: 'upload' */],
  plugins: [
    chunkedUploadPlugin({
      collections: ['media'],
    }),
  ],
})

That's it. The plugin:

1. Registers a client React provider via admin.components.providers that calls setUploadHandler for each listed collection
2. Adds a root endpoints entry at /api/uploads/chunk (auth-gated, streams chunk bodies to disk)
3. Injects an upload.handlers entry into each listed collection that reads the form envelope's clientUploadContext, reassembles chunks from disk, and returns the buffer for Payload's pipeline
4. Injects an inline progress bar into each listed collection's admin.components.edit.beforeDocumentControls slot — editors see live percentage / chunk count / MB-uploaded right above the Save button, not in a corner toast (0.1.4+)
5. Schedules a background cleanup loop in onInit to sweep abandoned chunk sessions

After installing or upgrading, regenerate the host's import map so Payload can resolve the plugin's admin components:

pnpm payload generate:importmap

Options

| option | type | default | description | |--------|------|---------|-------------| | collections | string[] | — (required) | Upload-enabled collection slugs to attach the chunked handler to | | chunkSizeBytes | number | 25 MiB | Per-chunk size. Keep below the smallest body limit you traverse (Cloudflare free/Pro: 100 MB) | | maxChunks | number | 400 | Per-session chunk cap. With defaults, caps total file size at ~10 GiB | | endpointPath | string | '/uploads/chunk' | Mounted under Payload's /api/ prefix | | tempDir | string | os.tmpdir() + '/payload-chunks' | Where chunks are staged between client upload and server-side reassembly | | cleanupIntervalMs | number \| false | 30 min | Background cleanup interval. false to disable | | sessionTtlMs | number | 60 min | Sessions older than this get swept by the cleanup loop | | progressBar | boolean | true | Inject the inline progress bar above the Save button. Set to false to keep only the success / error toast (e.g., if you ship your own progress UI) |

How it works

1. Editor picks a file in the admin
2. The plugin's client provider has registered a handler with Payload's useUploadHandlers().setUploadHandler({ collectionSlug, handler })
3. Payload's <Form> sees the registered handler and calls it BEFORE submitting
4. The handler slices the File into chunks and POSTs each to /api/uploads/chunk with X-Upload-{Session,Index,Total,Filename,Mimetype} headers
5. The plugin's endpoint streams each chunk body to <tempDir>/<session>/<index>.bin (no full-chunk buffering in memory)
6. After the final chunk lands, the handler returns { sessionId, total, mimetype } as the clientUploadContext
7. Payload's <Form> submits a tiny JSON envelope (no file bytes) to the collection's create endpoint
8. Payload's addDataAndFileToRequest sees the envelope and invokes the collection's upload.handlers[]
9. The plugin's collection handler reassembles the chunks into a Buffer and returns it as a Response
10. Payload feeds req.file.data from that Response and runs the rest of the upload pipeline unchanged

Chunks are sequential. Each chunk is retried up to 3 times with exponential backoff (250ms, 500ms) on network errors and 5xx. 4xx errors are permanent and surface immediately.

Caveats

• Next.js 16+ middleware silently truncates request bodies at 10 MB. If the host app has any middleware.ts whose matcher covers /api/<endpointPath> (default /api/uploads/chunk), Next will cap each chunk body at 10 MB regardless of how big the client sent — assembled file ends up corrupt and Sharp fails with "premature end of data segment". Fix: exclude the chunk endpoint from your middleware matcher, e.g. matcher: ['/((?!_next|api/uploads/chunk).*)']. The endpoint requires req.user internally, so middleware-level auth gating on it is redundant. Alternatively bump the global cap via middlewareClientMaxBodySize in next.config.js, but that's coarser. Pre-Next-16 hosts are unaffected.
• Realistic per-upload memory ceiling is far below the chunk cap. The 10 GiB ceiling (400 chunks × 25 MiB) is a per-session protocol cap, not an effective limit. Server-side, assembleSession concatenates all chunks into a single Buffer, hands it to Payload, then Payload's pipeline runs (Sharp resize, imageSizes, hooks) — peak memory ≈ 3–5× the file size for raster uploads. On a 4 GiB Lightsail instance, comfortable ceiling is ~500 MiB raster / ~2 GiB PDF or video (PDFs and MP4s bypass Sharp entirely). Tune maxChunks per deployment.
• Sharp memory pressure on huge rasters: a 500 MiB JPG can decode to multiple GiB. Sharp's default limitInputPixels: 268MP errors out on monsters instead of OOMing — keep it on. Consider rejecting raster uploads >100 MiB at the application layer if your instance is small.
• Single-instance only by default: chunks land on local disk. For multi-instance deployments, point tempDir at a shared volume (NFS, EFS) — but each session must hit the same instance for assembly. Sticky sessions or a session-routing layer needed.
• No resume on failure (v0.1): if the client connection drops mid-upload, the editor retries from chunk 0. Chunks already on disk are swept by the cleanup loop within sessionTtlMs.
• Auth gating: the chunk endpoint requires req.user (any authenticated Payload user). Same auth surface as the admin itself.

Interoperability with other upload-handler plugins

The collection-level handler runs on both the upload path AND the file-serve path (Payload calls upload.handlers from both). This plugin's handler:

• Serve path (no clientUploadContext in params): returns void. Default file serving — or any other handler's serve logic — runs.
• Upload path with our envelope shape (clientUploadContext matches {sessionId, total, mimetype}): assembles + returns the Response with the buffer.
• Upload path with a DIFFERENT envelope shape: returns void. Another plugin's handler is responsible for that envelope. The combination is safe — both plugins coexist as long as their clientUploadContext shapes don't collide.

Development

pnpm install
pnpm build    # tsc → dist/
pnpm dev      # tsc --watch

To test against a host project (e.g., a Payload site sitting next to this repo), use pnpm link --global here and pnpm link --global @assemblyltd/payload-chunked-upload in the host.

License

MIT