image-pump
v0.1.0
Published
Progressive responsive-image loading orchestrator: width ladder, concurrency limit, subscriptions, fallback upgrades without downgrades
Maintainers
Readme
image-pump
A framework-agnostic progressive responsive-image loading orchestrator for the browser.
It coordinates loading of the same image in multiple widths (a "width ladder", the srcset sizes your image backend produces) with:
- a concurrency limit (
maxActivePumps) and an internal load queue, - subscriptions ("I need image X at width W, notify me when usable"),
- fallback / upgrade semantics over the width ladder — a subscriber may be temporarily served a smaller width, then upgraded, but never downgraded,
- request deduplication: one image × one width = at most one network request,
- backpressure: after a slow load the pump keeps its slot busy for a couple of frames, so upgrade bursts cannot saturate the network channel and starve not-yet-visible images.
Zero runtime dependencies. TypeScript, dual ESM/CJS build.
Contents
- Motivation
- Why not native
srcset/sizes? - Core model
- Guarantees (the contract)
- Installation
- Quick start
- API
- Recipes
- Design notes
- Related libraries
- Development
- Roadmap
- License
Motivation
Typical setup: an image backend (imgproxy
& friends) can serve each attachment in any width on demand — the width is
encoded right in the URL processing options — so your app defines a fixed
ladder of widths (100, 200, … 800). A UI — gallery grid, lightbox, carousel —
renders many images at once, and each rendered box may need a different width.
Naive <img src> per box causes:
- duplicate requests for the same image at different widths,
- visible images competing with offscreen ones for the connection pool,
- janky upgrades when a bigger width replaces a smaller one mid-view.
ImagePump is the single point that owns "which (image, width) pairs are being
loaded / loaded / failed" and decides what to fetch next. The component layer
stays declarative: subscribe to what you need, render what you get.
Lazy loaders trigger requests, framework image components generate markup,
Suspense image caches deduplicate a single source — but none of them
orchestrates many consumers of the same image across a width ladder. To our
knowledge, no library combines progressive width upgrades, a shared concurrency
budget, per-(image, width) deduplication and no-downgrade delivery — that
combination is the reason ImagePump exists.
Why not native srcset/sizes?
The browser already picks a width from a srcset, prioritizes the fetch and
keeps it in the HTTP cache — for markup you control end-to-end, prefer it.
ImagePump earns its keep only when the consuming side is JavaScript:
- you need the loaded src itself — drawing to a
<canvas>, using it as a CSS background from JS, feeding it to WebGL —srcsetnever tells you which URL it chose; - many components share one image across widths — a gallery card at
100px and a lightbox at 800px want one shared load budget, deduplication
and upgrade policy, not two independent
<img>decisions; - you need delivery guarantees — the no-downgrade rule survives DOM
updates, re-subscribes and queue rotation;
srcsetre-evaluation on resize promises nothing like it; - you need to program the queue — concurrency limits, backpressure, prefetching the next carousel slide are not expressible in markup.
If none of these apply, stay with plain srcset — ImagePump will not beat
the browser's own scheduler.
Core model
ImagePump
├── state: Map<imageId, { [widthKey]: { src, status, subscriptions } }>
├── subscriptions: Array<{ imageId, image, width, widthKey, callback }>
└── activePumps: number- image — your domain object (e.g.
{ attachmentId, basename, extension }). - imageId — a stable cache key derived by
createImageId(see Design notes for what to include — and what to deliberately exclude). - width ladder — the ascending list of widths your backend can produce. Validated at construction time (dev mode only).
- status per
(imageId, width):pending → loading → loaded | failed.failedis terminal for the current pump lifetime (no re-fetch on re-subscribe; retries are a roadmap item).
Subscribe flow
subscribe(image, width, callback)registers a subscription and creates the(imageId, width)state slot if absent (srcis computed eagerly viacreateImageSrc).- If a free pump slot exists and the load is actually needed
(
needsLoad), loading starts immediately; otherwise the subscription sits in the queue and is picked bygetNextLoad()when a slot frees up. - On success,
publishSuccessnotifies the image's subscribers with the loadedsrc— respecting the no-downgrade rule (see below). On failure,onFailuresubscribers are notified the same way. Withimmediate: truethe best already-knownsrc(if any) is also delivered synchronously at subscribe time. - The returned function unsubscribes; it is idempotent (double call is safe).
needsLoad — when a request is skipped
A load for (imageId, width) is not started when this width or any width
above it is already loading or loaded:
- loading 400 → subscribing to 100 issues no request (100 will be upgraded to 400 when it arrives);
- the same
(imageId, width)subscribed twice → one request; - a
failedwidth is never retried.
getAnyLoaded — synchronous lookup
getAnyLoaded(image, width?, onlyLoaded?) returns the best already-known src
without waiting:
- exact width first;
- then the closest loaded width above (an acceptable upgrade), then below;
- then, unless
onlyLoaded, the closest loading width above/below — useful to render something that will be in cache soon; width: undefined→ the closest loaded/loading width from the top.
Use onlyLoaded: true when the consumer cannot tolerate a loading src (e.g.
it must not point <img> at a URL that is not in cache yet).
No-downgrade rule
A subscriber's rendered width may only grow. When a width finishes loading, subscribers that already hold an equal-or-bigger loaded width are not notified. This covers all the orderings (bigger resolved first, smaller resolved later; a smaller width loading after a bigger one finished, etc.).
Backpressure pause
When a load takes longer than IMAGE_PUMP_AFTER_LOAD_DELAY (2 frames at 60
FPS ≈ 34 ms), the pump keeps its slot busy for the same delay before taking
the next queued load. This prevents a cascade of upgrades from monopolizing
the channel right after a slow response.
Slot cleanup
A slot exists to serve subscriptions. When the last subscriber of a
pending slot goes away, the slot is unreachable garbage (only
subscriptions trigger loads) and is dropped immediately — long-lived
galleries do not accumulate dead slots. Loaded slots stay as a cache;
failed slots stay to keep failed terminal for re-subscribes.
dropState(image) drops everything about an image, including its live
subscriptions; in-flight loads finish silently.
Guarantees (the contract, enforced by the spec)
maxActivePumpsconcurrent loads at most; the queue drains automatically as slots free. A slot is always released (try/finally), so no failure path can permanently shrink the concurrency budget.- One network request per
(imageId, width); duplicates are never issued — including the "smaller width while a bigger one is loading" case. - Never downgrade a delivered source; never re-notify subscribers that already hold an equal or bigger loaded width.
unsubscribeis idempotent; a load completing after full unsubscribe is a no-op (does not throw).failedwidths do not restart on re-subscribe.- Subscriptions sharing the same
imageIdbut differentimagepayloads (e.g. different extensions) share one cache slot; the first started source wins for everyone (see Design notes). - Subscriber callbacks (
callbackandonFailure) are isolated: one throwing cannot break other notifications or the pump. - A pending slot that loses its last subscriber is garbage-collected; loaded slots stay as a cache, failed slots stay to keep guarantee 5.
dropState(image)wipes the image's state and live subscriptions; in-flight loads finish silently, stale unsubscribe functions become no-ops.
Installation
npm install image-pump
# or
yarn add image-pumpQuick start
import { ImagePump } from 'image-pump'
interface IAttachment {
id: string
basename: string
extension: 'jpg' | 'webp' | 'avif'
}
const widths = [100, 200, 300, 400, 500, 600, 700, 800] as const
const pump = new ImagePump<IAttachment, typeof widths[number], Omit<IAttachment, 'extension'>>({
maxActivePumps: 3,
widths,
// The cache key deliberately omits `extension` — see Design notes.
createImageId: ({ id, basename }) => `${id}/${basename}`,
createImageSrc: ({ id, basename, extension }, width) =>
`/api/attachments/${id}/${basename}/${width}.${extension}`,
})
// Somewhere in a component:
const unsubscribe = pump.subscribe(attachment, 400, (src) => {
imgElement.src = src // may first receive a smaller width, then be upgraded
})
// on teardown:
unsubscribe()
// Synchronous "what do we already have?":
const known = pump.getAnyLoaded(attachment, 400)API
new ImagePump<I, W, ID>(options)
| Option | Type | Notes |
| ------------------ | ------------------------------------- | ------------------------------------------------- |
| maxActivePumps | number | Concurrent load limit (queue drains automatically) |
| widths | readonly W[] | Ascending width ladder; validated in dev mode |
| createImageId | (image: ID) => string | Stable cache key |
| createImageSrc | (image: I, width: W) => string | URL factory |
| loadTimeoutMs | number | Stall kill-switch: fail a load after this many ms; 0 (default) disables |
Type parameters: I — full image identity used to build a src;
ID — identity used for the cache key (defaults to I). Splitting them lets
several src variants (formats) share one cache slot.
Instance members
subscribe(image, width, callback, options?)— returns the unsubscribe function. Options:immediate: true— deliver the best already-known source synchronously at subscribe time (without it a subscription only hears about loads that finish later);onFailure—(failure: { source, width }) => void, notified when a candidate width fails while this subscription could have received it, under the same no-downgrade rules as success (e.g. LQIP fallbacks).
prefetch(image: I, width: W): void— warm the cache without keeping a subscription (hover-preloads, next carousel slides).getAnyLoaded(image: I, width?: W, onlyLoaded?: boolean): string | undefineddropState(image: ID): void— wipes the image's state and live subscriptions; in-flight loads finish silently.createImageSrc(image: I, width: W): string— exposed factorystate— read-only introspection:ReadonlyMap<imageId, { [width]: { src, status, subscriptions } }>subscriptions— read-only live subscription listactivePumps: number
Recipes
React hook (canonical usage from the project of origin)
import { useEffect, useMemo, useState } from 'react'
export function useImagePumpedSource(
pump: ImagePump<IAttachment, ImageWidth, Omit<IAttachment, 'extension'>>,
image: IAttachment | undefined,
width: ImageWidth | undefined,
): string | undefined {
const [source, setSource] = useState<string | undefined>(
() => image ? pump.getAnyLoaded(image, width) : undefined,
)
useEffect(() => {
if (!image) return
if (!width) {
setSource(pump.getAnyLoaded(image, undefined))
return
}
// immediate: true re-delivers the best known source on every width change
return pump.subscribe(image, width, setSource, { immediate: true })
}, [pump, image, width])
return source
}Details worth copying:
- initial
useStateseeds synchronously fromgetAnyLoaded— the first paint can already use a pumped (possibly smaller) width instead of a spinner; immediate: truere-delivers the best known src on every width change, and later loads upgrade it through the samesetSourcecallback;- the effect returns the unsubscribe function directly.
Prefetching
const startHoverPreload = () => pump.prefetch(nextAttachment, 800)The load starts immediately (subject to the shared concurrency budget) and the loaded slot stays in the cache for the actual subscription — no bookkeeping to undo. A prefetch of a width that is already superseded by a bigger loading one leaves no trace at all.
SSR / first render without a pump
On the server there is no pump; render the URL the cache would eventually hold, or an LQIP:
const src = pump.getAnyLoaded(image, width, true) || pump.createImageSrc(image, width)onlyLoaded: true avoids handing the browser a loading src that may never
finish before hydration.
Bots and crawlers
Bots get the direct src without waiting for subscriptions and pump bookkeeping:
const src = isBot
? pump.createImageSrc(image, width)
: pump.getAnyLoaded(image, width)Design notes
createImageId deliberately omits the image format
One attachment + width = one cache slot, regardless of the requested format
(webp/avif/jpg). The first started format "wins": its src is written into
the state and reused for every later subscription to the same
imageId + widthKey. This is acceptable when the format is stable within a
session (server-side negotiation via SSR/cookies), so a duplicate download of
another format is never needed. A test pins this behavior.
Why loading counts as "good enough" in getAnyLoaded
A loading src will be in the HTTP cache by the time it finishes, so pointing
an <img> at it merely defers the decode, not the network. Pass
onlyLoaded: true to opt out.
Callbacks are untrusted, errors are isolated
Consumer callbacks (callback, onFailure) can throw without breaking
neighboring notifications or the pump's accounting; the load path releases
its slot in a finally, so no failure — a rejected custom loader included —
can permanently shrink the concurrency budget. The loadTimeoutMs option
adds a stall kill-switch: a load older than the timeout is failed instead of
holding its slot forever.
Queue ordering
getNextLoad currently picks the first subscription that still needs a load
(FIFO by subscription order). Prioritization (visible-first, recency/LIFO) is
the main roadmap item — the call site is a single method.
Related libraries
Each of these covers one axis of ImagePump — none combines a width ladder, a
shared concurrency budget, per-(image, width) deduplication and
no-downgrade delivery:
- unpic-img — the standard
multi-framework
<Image>for CDN URLs:srcset/sizesmarkup, browser scheduling; no orchestration for JS consumers. - lazySizes + RIAS/optimumx —
substitutes a
{width}placeholder in the URL with the measured element width; per-element, so no shared budget, no upgrade policy, no programmaticsrc. - PixiJS Assets and PreloadJS LoadQueue — asset queues with a concurrency limit; they load lists of URLs, not "the best width for each subscriber".
- Cornerstone.js progressive loading — smaller/lossy first, then an upgrade, over a priority pool: the same semantics, but inside a DICOM-viewer stack.
URL builders such as @imgproxy/imgproxy-js-core
compose with ImagePump (createImageSrc can delegate to them) instead of
competing.
Rule of thumb: markup consumers — srcset + a URL builder or unpic-img;
JavaScript consumers sharing one image across widths — that is the gap
ImagePump fills.
Development
The development setup, the full verification chain (lint → typecheck → test → build → dist checks), the contribution conventions and the release process are described in CONTRIBUTING.md. User-facing changes are tracked in CHANGELOG.md.
Roadmap
- [ ] Retry strategy for
failedwidths (todayfailedis terminal); it should distinguish network failures fromdecode()failures, which are often transient. - [ ] Queue prioritization in
getNextLoad(visible-first / recency). - [ ] Configurable loader (replace the default
createImagePromise, e.g. forfetch+ blob URLs or canvas-driven sources; fold incrossOrigin/fetchPriorityknobs — the timeout and error contract already exist).
