sharp-dicer

Stream a multipart upload straight through sharp into a set of fixed-size images, and pipe each resized image directly to Amazon S3 — without ever buffering the whole file in memory or writing it to disk.

The incoming request is parsed with busboy, each image part is read once and fanned out to every configured size via sharp's streaming clone(), and each output is uploaded with the AWS SDK v3 @aws-sdk/lib-storage multipart Upload (backpressure- and retry-aware).

This is a native ES module (import) and requires Node.js 24+.

Install

npm install sharp-dicer

Usage

import http from 'node:http'
import sharpDicer from 'sharp-dicer'

// Build the middleware once, at startup — it holds a reused S3 client.
const handleUpload = sharpDicer({
  s3: { region: 'us-east-1', bucket: 'my-bucket' }
})

// Then, per request — Promise style:
http.createServer(async (request, response) => {
  const output = {
    base_key: `users/${userId}/avatar`,
    images: {
      'large.jpg': { width: 1024, height: 1024 },
      'thumb.jpg': { width: 128, height: 128 }
    }
  }

  try {
    const { handled } = await handleUpload(request, output)
    if (!handled) return response.writeHead(415).end('no supported image')
    response.writeHead(201).end('uploaded')
  } catch (err) {
    response.writeHead(500).end(String(err))
  }
})

The uploaded objects land at ${output.base_key}/${name} for each key in output.images (e.g. users/42/avatar/thumb.jpg).

Callback style

Pass then / unless callbacks instead of awaiting and the middleware returns undefined:

handleUpload(
  request,
  output,
  (err) => {                       // called once, after ALL uploads finish
    if (err) return response.writeHead(500).end(String(err))
    response.writeHead(201).end('uploaded')
  },
  () => response.writeHead(415).end('no supported image')
)

API

sharpDicer(config) → middleware

config:

| field | required | description | |---|---|---| | s3.region | yes | AWS region | | s3.bucket | yes | destination bucket | | client | no | reuse an existing @aws-sdk/client-s3 S3Client | | allowableImageFormats | no | array of accepted part content-types (default: jpeg/jpg/png) | | jpeg | no | options forwarded to sharp's .jpeg() (e.g. { quality: 82 }) | | uploadOptions | no | { queueSize, partSize } to tune S3 multipart concurrency |

A settings provider exposing .get('aws') is also accepted for backward compatibility with the original nconf-style config.

middleware(request, output[, then, unless])

• request — the incoming readable request (must carry headers).
• output — { base_key, images: { [name]: { width, height } } }.

A request is either handled — at least one supported image part was resized and uploaded — or unhandled — it was not a multipart request carrying a supported image. Non-image parts of an otherwise valid upload (plain form fields) are skipped, not rejected.

Promise mode (omit the callbacks) — returns Promise<{ handled: boolean }>, resolved only after every upload completes, or rejected on the first error.

Callback mode (pass then and/or unless) — returns undefined:

• then(err) — called exactly once after every upload completes; err is set on the first error (error-first, node-style).
• unless() — called instead of then when the request is unhandled.

then and unless are mutually exclusive: exactly one of them fires per request (or then(err) on error).

Testing

Fast, hermetic unit tests (no network):

npm test

End-to-end tests against a real S3 API via localstack (requires Docker). They upload a generated image through the middleware and assert that every resized object lands in S3 with the right format and dimensions:

npm run localstack:up      # start localstack (docker compose)
npm run test:integration
npm run localstack:down    # stop it

The integration suite skips itself automatically if no S3 endpoint is reachable, so it's safe to run anywhere. Override the endpoint with S3_ENDPOINT if needed.

Notes

• Requires Node.js 24+ (see .nvmrc); run nvm use in the project root.
• Auto-orientation (.rotate() with no args) is applied from EXIF before resize.
• All outputs are encoded as JPEG (ContentType: image/jpeg).

License

MIT © Jason Sperske