@pmndrs/upscaler

AMD FidelityFX Super Resolution (FSR) brought to three.js WebGPURenderer as raw WGSL compute passes, with an interactive test bench.

Three ships an official FSR1Node — spatial-only upscaling. This package goes further: three's WebGPU renderer already produces every temporal input FSR2/3 needs (depth, per-pixel motion vectors via the velocity node, jitterable projections), so we can run a temporal upscaler — the architecture behind FSR 2/3, DLSS, and XeSS — which reconstructs detail spatial upscalers can't, and anti-aliases for free.

WebGPU only. The passes are hand-written WGSL dispatched straight on the renderer's GPUDevice — no TSL, no WebGL fallback. This is deliberate: the goal is a performance-first pipeline with sources that read like the FidelityFX originals.

Install

npm install @pmndrs/upscaler three

WebGPU only — needs a WebGPU-capable browser (Chrome/Edge 113+) and three r184+ (a peer dependency). There is no WebGL fallback.

▶ Live demos: pmndrs.github.io/upscaler — 11 interactive examples: spatial vs temporal, the aliasing-torture scene, transparency + reactive masks, the composable TSL node, SSGI/SSR upscaled in one post graph, and more.

Using the upscaler

The recommended integration is the TSL node — drop it in as the output of a THREE.PostProcessing graph and it renders your scene at a reduced resolution and upscales it back, jitter and all:

import * as THREE from 'three/webgpu';
import { upscaleScene, QualityMode } from '@pmndrs/upscaler';

// The upscaler remains linear/HDR. Choose presentation independently.
renderer.toneMapping = THREE.ACESFilmicToneMapping;
renderer.outputColorSpace = THREE.SRGBColorSpace;

const post = new THREE.PostProcessing(renderer);
post.outputNode = upscaleScene(scene, camera, { quality: QualityMode.Quality });

renderer.setAnimationLoop(() => post.render());

That's the whole thing — no manual jitter, MRT, or velocity wiring. upscaleScene renders the scene in-graph as an FSR input, so the sub-pixel jitter lands on it and you get real reconstruction (not just a smart blur).

Composing with other effects. If you already render a reduced-resolution G-buffer feeding SSGI / SSR / GTAO, feed those texture nodes to the composable upscale() node and it upscales the composited result — the scene, effects, and upscale are one post graph:

import { pass, mrt, output, velocity } from 'three/tsl';
import { upscale } from '@pmndrs/upscaler';

const scenePass = pass(scene, camera);
scenePass.setMRT(mrt({ output, velocity }));
scenePass.setResolutionScale(0.5); // render at half res per axis
// …composite your SSGI/SSR onto the reduced-res color here…

post.outputNode = upscale(
    composedColor,
    scenePass.getTextureNode('depth'),
    scenePass.getTextureNode('velocity'),
    camera,
    { ratio: 2, reactive /* optional mask */, exposureTexture /* optional */ },
);

Color-only input with no motion data? upscaleSpatial(color) runs the spatial (FSR1) path — no history, no depth/velocity. The live demos cover every node path (examples 07–11).

Low-level API

When you are not in a post-processing graph — compositing in your own render-target loop — drive the imperative Upscaler directly. (UpscalePass bakes this exact MRT/jitter/velocity/present recipe into a renderer-agnostic drop-in if you want it done for you.)

import { Upscaler, QualityMode } from '@pmndrs/upscaler';
import { velocity, mrt, output } from 'three/tsl';

const upscaler = new Upscaler({ renderer });
upscaler.init();
upscaler.configure({
    displayWidth: canvas.width,
    displayHeight: canvas.height,
    qualityMode: QualityMode.Quality, // renders at 1/1.5 res per axis
    path: 'temporal',
});

// Scene render target: color + velocity MRT at *render* resolution.
const rt = new THREE.RenderTarget(upscaler.renderWidth, upscaler.renderHeight, {
    count: 2,
    type: THREE.HalfFloatType,
    depthTexture: new THREE.DepthTexture(upscaler.renderWidth, upscaler.renderHeight),
});
// Motion vectors must be jitter-free — feed the velocity node the upscaler's
// unjittered projection (stable instance, refreshed per frame).
velocity.setProjectionMatrix(upscaler.unjitteredProjectionMatrix);

//* Per frame
upscaler.beginFrame(camera); // applies sub-pixel jitter (setViewOffset)
renderer.setMRT(mrt({ output, velocity }));
renderer.setRenderTarget(rt);
renderer.render(scene, camera);
renderer.setRenderTarget(null);
renderer.setMRT(null);
upscaler.endFrame(camera); // clears the jitter offset

upscaler.dispatch(
    { color: rt.textures[0], depth: rt.depthTexture, velocity: rt.textures[1], deltaTime },
    camera,
);
// upscaler.outputTexture is a display-resolution linear/HDR texture. Present it
// through the renderer's normal output transform or continue post-processing it.

Runtime knobs live on upscaler.settings (sharpness, maxAccumulation, exposure, debugView) and take effect next frame. upscaler.resetHistory() drops accumulation on camera cuts.

How it works

Why temporal upscaling works

Each frame the projection is offset by a sub-pixel jitter (Halton(2,3) sequence, 8·ratio² phases — at 2× upscale, 32 frames sweep 32 distinct sample positions per pixel). A static scene therefore delivers a super-sampled image over time; the upscaler's job is to integrate those samples — and to know when not to (movement, disocclusion, shading change), falling back to spatial reconstruction there.

The pipeline

                       render res                                    display res
              ┌──────────────────────────┐              ┌────────────────────────────────┐
   depth ────▶│ dilate                   │              │                                 │
   velocity ─▶│  nearest-depth 3×3       │─ motion ────▶│ accumulate                     │
              │  motion + linear depth   │─ depth ─┐    │  jitter-aware Lanczos2 upsample │
              └──────────────────────────┘         │    │  Catmull-Rom history reproject  │──▶ history
              ┌──────────────────────────┐         │    │  YCoCg variance clip            │      │
   history ──▶│ depth clip               │◀────────┘    │  disocclusion-weighted blend    │      ▼
   depth  ───▶│  disocclusion mask       │─ mask ──────▶│                                 │   ┌──────┐
              └──────────────────────────┘              └────────────────────────────────┘   │ RCAS │──▶ output
                                                                                              └──────┘

1. Dilate — per render pixel, find the nearest depth in a 3×3 ring and take that texel's motion vector (foreground silhouettes drag their motion), storing linearized view depth.
2. Depth clip — reproject into last frame's dilated depth; when the previous surface was meaningfully nearer, the pixel was occluded and its history is poisoned → disocclusion mask.
3. Accumulate — the core. Upsamples the current frame with a jitter-aware Lanczos2 kernel, samples history through the motion vector with a 9-tap Catmull-Rom, rectifies history against the current neighborhood's YCoCg variance box, and blends with a per-pixel accumulation counter (stored in history alpha) so fresh regions converge fast and stable regions stay smooth. Runs in invertible-tonemap space so HDR fireflies can't dominate.
4. RCAS — FSR's analytically-bounded contrast-adaptive sharpener counteracts the mild softness of temporal integration.

The spatial path (path: 'spatial') is a faithful FSR1 port: EASU's edge-direction-rotated, anisotropically-stretched 12-tap Lanczos kernel, then RCAS. No history, no motion vectors — also the fallback story for content that can't produce velocity.

Full per-pass details and deviations from the FidelityFX reference: src/shaders/README.md. For the measured story of how this implementation relates to real FSR 3.1.5 — what matches, what was re-derived into cheaper forms, and the benchmark evidence — see PARITY.md.

Integration approach

Three doesn't expose its WebGPU internals publicly, so the upscaler grabs renderer.backend.device and the GPUTexture handles behind render-target attachments (internal/threeWebGPU.ts documents exactly which internals we touch and throws loudly if a three upgrade changes them). Compute passes are encoded on our own GPUCommandEncoder and submitted between three's scene render and the presentation draw — queue order guarantees correctness with zero synchronization code. The final image lands in a three StorageTexture so presenting it is ordinary three code.

Temporal guides

Dilated motion, dilated depth, and disocclusion are frame properties, not upscaler properties — every temporal effect upstream (SSGI/SSR temporal reprojection, denoisers, any TAA-class pass) needs them and usually re-derives worse versions privately. The upscaler publishes its internal working set as the temporal guides bundle (upscaler.guides, ordinary three textures), and the frame can be driven split so the geometry guides exist before the final color does:

upscaler.dispatchGuides({ depth, velocity }, camera); // right after the G-buffer
// … effects sample upscaler.guides.dilatedMotion / .disocclusion / .dilatedDepth …
upscaler.dispatchUpscale({ color, deltaTime }, camera); // finish the frame

An app that never upscales can run path: 'guides' for the geometry products alone. Reactivity is bidirectional: an explicit mask merges (per-pixel max) with the auto-generated one, and effects can write into guides.reactive mid-frame. The standalone MomentsPass rounds out the bundle — per-pixel (E[x], E[x²]) of a configurable scalar (linear luma or YCoCg Y) over any float texture plus one coarse level, the statistics half an SVGF-class denoiser needs, with no beauty/exposure assumption baked in.

The same surface exists declaratively for THREE.PostProcessing graphs: temporalGuides(depth, velocity, camera) publishes the bundle as texture nodes (guides.getTextureNode('disocclusion')), and upscale(color, depth, velocity, camera, { guides }) shares one computation — the guides dispatch runs as soon as the G-buffer has rendered, in-graph effects consume the products, and the upscale finishes the split frame.

Per-product contracts (format, space, resolution, latency) are documented on the TemporalGuides type and in TEMPORAL-GUIDES-SPEC.md; examples/12-temporal-guides (raw) and examples/13-guides-node (TSL) are the live references. The contract is accepted: an external SSGI/SVGF consumer swapped its private temporal front-end for this bundle and measured bit-identical still-camera stability (spec M6). The TSL surface stays marked @experimental — the same contract, but its graph wiring has only our own verification so far.

Status

The pipeline is feature-complete and GPU-verified: spatial (FSR1) and temporal paths, luminance-stability locks, auto-exposure (+ external and host pre-exposure inputs), multi-scale shading-change detection, reactive masks (explicit + auto-generated), RCAS with opt-in denoise, imperative UpscalePass, and the composable TSL nodes (upscale / upscaleScene / upscaleSpatial). The temporal guides surface (above) has passed its cross-repo acceptance; only its TSL node stays experimental. A benchmarking program A/B-compared this implementation against source-style FSR 3.1.5 pass graphs on-GPU; the adopted results and remaining divergences — with measurements — are written up in PARITY.md.

Deliberately not planned:

• Frame generation (the other half of "FSR3") — needs swapchain-level frame pacing browsers don't expose.
• MSAA input — FSR's temporal path is the anti-aliaser; a multisampled input is redundant and can't bind to the compute passes.
• Perf-only micro-optimizations (textureGather tap packing, f16 arithmetic, bind-group caching) — each adds correctness risk to a core path with no image-quality gain; deferred until performance is an actual bottleneck on real content.

Package layout

src/
  Upscaler.ts      — public API / pass orchestration
  types.ts             — quality modes, config, settings
  math/                — Halton, jitter sequencing, resolution presets (unit-tested)
  shaders/             — WGSL sources as TS modules + assembler (unit-tested)
  internal/            — device access, constants UBO, pass + timestamp helpers
bench/                 — Vite test bench (npm run dev)

Develop

Clone the repo, then:

npm install
npm run dev        # interactive bench   → http://localhost:5199
npm run examples   # example gallery      → http://localhost:5300
npm test           # unit tests (GPU-free)
npm run typecheck  # tsc --noEmit
npm run lint       # eslint
npm run build      # library build → dist/

The bench (in bench/) renders an aliasing-hostile scene and lets you flip between native rendering, bilinear upscaling, FSR1 spatial, and FSR3 temporal — with quality presets, sharpness control, debug views, and per-pass GPU timings. The example gallery is what's deployed to GitHub Pages.

Releasing

Publishing is automated — just push to main. A GitHub Action reads your Conventional Commit messages since the last release and, if any warrant one, bumps the version, publishes to npm, and pushes the release commit + tag back. Auth is OIDC Trusted Publishing (no tokens, provenance attached); npm publish runs the full lint/typecheck/test/build gate first.

| Commit type on main | Bump | Example | | ----------------------------------------- | ----------------- | --------------- | | feat: … | minor | 0.1.0 → 0.2.0 | | fix: … / perf: … | patch | 0.1.0 → 0.1.1 | | feat!: … / BREAKING CHANGE: | major* | 0.1.0 → 0.2.0* | | docs: / chore: / ci: / refactor: …| no release | — |

* While in 0.x, a breaking change bumps minor (not 1.0.0) so a stray break can't cut a major. Edit scripts/release-version.mjs to change the policy.

Manual / prerelease override. Set an explicit version yourself and the Action publishes exactly that instead of auto-bumping — prereleases route to their own dist-tag so they never overwrite latest:

npm version prerelease --preid next   # 0.2.0-next.0
git push origin main --follow-tags    # → publishes to the `next` tag

| You set | Publishes to | Install | | ----------------------------- | ------------ | ------------------------------ | | 0.2.0 (stable) | latest | npm i @pmndrs/upscaler | | 0.2.0-next.0 | next | npm i @pmndrs/upscaler@next | | 0.2.0-beta.0 | beta | npm i @pmndrs/upscaler@beta | | 1.0.0-rc.0 | rc | npm i @pmndrs/upscaler@rc |

References

• FidelityFX Super Resolution 2/3 (GPUOpen) — algorithm & source (MIT)
• ffx_fsr1.h — EASU/RCAS reference the WGSL ports follow
• "Filmic SMAA / temporal reprojection" (Jimenez, SIGGRAPH 2016) — Catmull-Rom history filtering
• "Temporal Reprojection Anti-Aliasing" (Playdead) — variance clipping
• three.js TRAANode — jitter/velocity integration pattern this package mirrors

Credits

Built by Dennis Smolek. Maintained under the Poimandres collective.

Based on AMD's FidelityFX Super Resolution — this package ports its MIT-licensed EASU/RCAS shaders and follows the FSR2/3 temporal-upscaling architecture. "FSR" and "FidelityFX" are AMD's; this is an independent, unaffiliated implementation for three.js.

License

MIT — see LICENSE. The EASU/RCAS shaders derive from AMD's MIT-licensed FidelityFX Super Resolution; AMD's copyright notice is included in the license file.