pgfjs

A clean-room, dependency-free PGF (Progressive Graphics File) codec for Node.js. Give it the bytes of a .pgf file, get back RGBA pixels — and write lossless PGF back out.

import decode from '@kevcom71/pgfjs';

const { width, height, data } = decode(await readFile('thumb.pgf'));
// data is a Uint8ClampedArray, RGBA, 8 bits per channel, width * height * 4

• License: MIT — see LICENSE
• Runtime dependencies: none
• Node: >= 18 · Module format: ESM

PGF is a wavelet image codec by Christoph Stamm. It is not widely used as an interchange format, but digiKam stores its thumbnail cache as PGF blobs, so anything that wants to read that cache needs a decoder. The reference implementation, libpgf, is LGPL-2.1-or-later; this project exists so that permissively-licensed software can read those files without linking it.

Why this repository is unusual

The format's entropy-coded bitstream is not documented anywhere. The two published papers specify the container, the color transform and the wavelet lifting steps, and then say outright: "We omit the discussion of the decoder." That omission covers the entire entropy coder — the part standing between a byte buffer and an image.

So this decoder was not written from a specification. The container and transforms were; everything below them was recovered by measurement: construct an input with known contents, encode it with a black-box encoder binary, and read the resulting bytes knowing exactly what they must represent.

Every rule the decoder applies is recorded with the evidence that establishes it in docs/SPEC-DIGEST.md §7.5 — including the hypotheses that were tried and later retracted. If you are evaluating this code, that file, not this one, is the substantive document.

The result is written up as a complete format specification, for decoding and encoding both: docs/PGF-FORMAT.md. It is language-agnostic and MIT licensed, so a codec can be implemented from it in any language without touching this code. If you came here to read PGF rather than to use this package, start there.

To read it in a browser instead, the same specification is generated as a standalone offline page: docs/pgf-bitstream.html — download and open it, no server or network needed. The Markdown is the source of truth and the HTML is a view of it; the test suite fails if the two disagree.

Two rules kept the result honest:

1. Never fill a gap with a plausible guess. An unmeasured value raises a typed PgfUndeterminedError instead. This is load-bearing for the clean-room claim — see below.
2. A rule is not believed until it predicts data it was not fitted on. The RLR k-rule's d = 29 boundary was predicted before it was measured; the alpha layout rule was derived at 16×16 and confirmed at 32×32.

If you are assessing the provenance of this code, read CLEANROOM.md first. It carries a disclosure about how the code was authored that materially affects how much you should trust it, and it is not optional reading.

Status

| | | |---|---| | Real digiKam thumbnails decoded | 11 / 11 | | Decoded files matching a reference decoder | 18 / 18 byte-identical | | Encoder output read back by libpgf | 360 / 360 bit-exact | | Lossless round-trips bit-exact | 24 / 24 | | RGBA encoding, q0–q31, vs the reference encoder | 960 / 960 coefficient-exact | | Test suite | 617 passing | | Pixel formats, our encode → libpgf decode | 210 / 210 bit-exact at q0 | | Pixel formats, libpgf encode → our decode | 210 / 210 bit-exact at q0 | | Fuzzing | 50,000 iterations, zero crashes — what npm run fuzz runs by default | | Coding variants still unmeasured | none | | Quantizer inputs still undetermined | one deadzone boundary — |v| = 359 at divisor 256, flagged in src/encode.js. Every divisor the quantizer reaches at 8-bit depths, 2 through 256, has its threshold probed; above shift 8 the rule — including the reference's 32-bit overflow from shift 29 — is established by coefficient agreement rather than per-divisor probes |

"Bit-exact" means: build an image, encode it with a third-party encoder, decode it here, compare — every byte identical. For a lossy format that is only possible at quality 0, which is why the lossless cases carry most of the verification weight. One wrong sign bit negates a real coefficient and shows up immediately.

Install

npm install @kevcom71/pgfjs

Source: gitlab.com/KevinHughes/pgfjs

Versioning

4.0.0 is the first release published to a registry, and the version numbers continue this repository's own sequence rather than restarting. Tags run from v1.0.0 through v3.10.0 and are public here; earlier versions were real releases that simply never went to npm. Renumbering the first publish to 1.0.0 would have made that string name two different code states, so it was not done.

Why a major bump. Two changes earn it. The package is now scoped — @kevcom71/pgfjs, where it was previously pgfjs — and container version 0x06 is fully supported, which means decode() now returns pixels for files that used to throw PGF_UNSUPPORTED_VERSION. Any caller depending on that refusal sees different behaviour, and that is a breaking change however welcome it is.

What the version commits to is the surface in types/index.d.ts — thirteen exports, pinned by a test so widening them takes a deliberate edit. The runtime exports rather more so the tools and tests can reach the parser internals; those are not API and may change without a major bump.

The specification carries its own version, unrelated to this one. docs/PGF-FORMAT.md is at 1.19 and counts revisions of the document, not of the code.

Usage

Decode to pixels

import decode from '@kevcom71/pgfjs';
import { readFile } from 'node:fs/promises';

const image = decode(await readFile('thumb.pgf'));

image.width   // number
image.height  // number
image.data    // Uint8ClampedArray, RGBA, width * height * 4 bytes

Input may be a Uint8Array, a Node Buffer, or an ArrayBuffer holding one complete PGF file.

Encode

import { encode } from '@kevcom71/pgfjs';

const pgf = encode({ width, height, data }, { quality: 0, levels: 3 });
// data is RGBA, 8 bits per channel, width * height * 4 bytes

encode({ width, height, data })   // channels inferred from data.length:
                                  //   w*h   -> grayscale
                                  //   w*h*3 -> RGB
                                  //   w*h*4 -> RGBA

Any other pixel format is selected explicitly, and is never inferred — bitmap and indexed are one byte per pixel, which is also grayscale's length, and lab and lab48 have exactly rgb's and rgb48's lengths, so guessing between them is exactly the kind of choice this package refuses:

encode(image, { colorModel: 'cmyk' })              // or rgb48, cmyk64, rgb12, rgb16
encode(image, { colorModel: 'lab' })               // or lab48 — no color transform
encode(image, { colorModel: 'untransformed5' })     // or untransformed6 — modes 5 and 6,
                                                    // 1, 3 or 4 untransformed 8-bit channels
encode(image, { colorModel: 'gray16' })            // data is a Uint16Array, w*h
encode(image, { colorModel: 'gray31' })            // data is a Uint32Array, w*h, 0..2^31-1
encode(image, { colorModel: 'indexed', palette })   // quality 0 only
encode(image, { colorModel: 'bitmap' })             // quality 0 only
encode(image, { roi: true })                        // region-of-interest coding, version 0x7E

roi: true writes a region-of-interest file: each level's coefficients are partitioned into independently decodable blocks instead of into 16384-coefficient macroblocks. Nothing else changes, so the pixels do not either — an ROI and an ordinary encode of the same input decode to the same samples through both this decoder and the reference. It is refused for a geometry that clamps to zero pyramid levels, because the reference codec crashes on such files; encode without roi there and you get the same bytes but for the version byte.

gray16 and gray31 are single-channel and wide, so decode gives them back as a Uint16Array and a Uint32Array of w*h samples rather than expanding them to RGBA8 — that would throw away 8 and 23 bits per sample and report it as a successful decode. They are never inferred either: a typed array of w*h elements has exactly grayscale-8's length. Note that "31 bits per pixel" means bpp 32 with 31 used bits; a sample of 2^31 or more is refused rather than truncated, as is content whose coefficients need more than the 31 bit-planes the format can express (the reference encoder writes a file its own decoder cannot read there).

Qualities 0–31, for every format whose samples are magnitudes. Quality 0 is lossless; 1–31 quantize, and from quality 4 the encoder also downsamples chroma and alpha to half resolution — except in RGB12 and RGB16, which keep every channel at full resolution. Those two, and the three single-channel sample formats (grayscale, gray16, gray31), also skip the step in the shift constant K at quality 4, so their quantization shift is one greater from there up; it is measured per format, seven of the sixteen — modes 5 and 6 skip both as well. All three encoder-side rules that quality ≥ 4 needs are now measured: the pyramid depth clamp, the quantizer's rounding and deadzone, and the downsampling filter (a 2×2 box average rounded down). Against the reference encoder over 9 geometries — odd dimensions included — × 2 patterns × q4–q9 × 1–3 levels, output is 324 / 324 coefficient-exact and 324 / 324 pixel-identical through the reference decoder. See docs/SPEC-DIGEST.md §7.5.77–79.

Qualities 10–18 were added later and measured the same way, over 3 geometries × 2 patterns × q10–q18: 54 / 54 coefficient-exact against the reference encoder, 54 / 54 pixel-identical through the reference decoder, and 54 / 54 for our decode of the reference's own files. The shift rule K = q - 2 simply continues. See §7.5.82.

Qualities 19–31 were added after that, and reaching them needed a change of method rather than a bigger probe. A deep coefficient's magnitude is bounded by the sample range, so 31-bit samples survive a shift near 30 where 8-bit samples survive 8 — quality 28 is measured on a 64 KiB image, where doubling the probe per quality would have needed 330 TiB to reach 31. 31 is the format's real ceiling: the field is a UINT8, but the reference clamps any higher request to 31, so nothing above it is expressible. Qualities 29–31 additionally required emulating a defect in the reference — it computes its deadzone threshold in 32-bit signed arithmetic, which overflows at a divisor of 2^29 and wraps negative, switching the deadzone off. See §7.5.113.

Encoded output round-trips bit-exactly and matches libpgf's own compressed size to a mean of 1.005× on files where both make the same coding choices. See Encoder verification for what that does and does not establish.

Identify a file without decoding it

The container is cheap to parse, so you can reject unsupported input before committing to a decode:

import { readHeader } from '@kevcom71/pgfjs';

const info = readHeader(buffer);
info.header.width;    // number
info.header.height;   // number
info.header.nLevels;  // number
info.header.quality;  // 0 = lossless

Errors

Errors are typed and carry a stable, machine-readable code. Branch on err.code, never on message text.

| Class | Meaning | |---|---| | PgfFormatError | The bytes are malformed, corrupt, truncated, or self-contradictory. | | PgfUnsupportedError | Well-formed PGF, but uses a feature outside the implemented subset. | | PgfUndeterminedError | The bytes need a part of the format that has not been determined. Deliberately distinct from "unsupported": it means the format is not pinned down here, not we chose not to implement it. |

import decode, { PgfError } from '@kevcom71/pgfjs';

try {
  return decode(bytes);
} catch (err) {
  if (err instanceof PgfError) return null;   // not a usable PGF file
  throw err;                                  // a real bug — do not swallow
}

Two guarantees, both tested:

• Error ordering. Malformed input is reported as PgfFormatError and out-of-subset input as PgfUnsupportedError, both before any decoding work.
• No partial success. decode() either throws a typed error or returns a complete width * height * 4 buffer. It will never hand back a short buffer, or pixels the file did not encode. A file that declares an image but carries no macroblocks is a truncation error, not a blank image. { maxLevel: k } is not an exception to this: it returns a complete smaller image at the reduced width/height it reports, and it is held to the same strictness — the bytes of the entries it consumes must all be present, and their segment count must match the geometry exactly, or it throws.

What is supported

| Feature | Status | |---|---| | RGBA, 32 bpp, 4 channels (mode 17) | Yes — bit-exact verified | | RGB, 24 bpp, 3 channels (mode 3) | Yes — bit-exact verified | | Grayscale, 8 bpp, 1 channel (mode 1) | Yes — bit-exact verified. Its lossy shift was CORRECTED: mode 1 does not take the quality-4 K step, see below | | Container versions 0x36, 0x76 | Yes — verified exact vs a reference decoder | | Container version 0x06 | Yes, fully. Two behaviours differ from the other containers and both are specified: HL/LH are interleaved on a joint 4x4 scan, and 1-bpp images are coded as PACKED BYTES rather than one bit per pixel. All ten sample files decode byte-exactly, including 393x501 RGBA and both bitmaps | | Quality 0 (lossless) | Yes — bit-exact | | Qualities 1–9 (lossy) | Yes — all measured byte-exact | | Qualities 10–18 (lossy) | Yes — 54 / 54 vs a reference decoder | | Qualities 19–31 (lossy) | Yes — the shift rule is solved from the reference encoder's own coefficients and holds to 31. Decoding needs only that rule: the reference's deadzone overflow above shift 28 changes an encoder's zero/non-zero decision and nothing a decoder does. See §7.5.103–7.5.113 | | Any number of pyramid levels | Yes | | nLevels = 0 (raw, uncompressed planes) | Yes — bit-exact verified | | Odd and non-power-of-two dimensions | Yes | | Chroma/alpha subsampling at quality ≥ 4 | Yes — and per format: RGB12 and RGB16 are NOT subsampled, and it is inert for the single-channel formats | | Post-header user data | Skipped correctly | | Post-header color table | Returned verbatim for indexed files; skipped otherwise | | Encoding, qualities 0–31 | Yes — 960 / 960 coefficient-exact vs the reference encoder, RGBA over 5 geometries × 2 patterns × every quality × 3 level counts (tools/pgfa-quality-check.js) | | Bitmap, 1 bpp, 1 channel (mode 0) | Yes — bit-exact against the reference in both directions. Two distinct codings: 0x36/0x76 store one bit per pixel as the sample's low bit; 0x06 PACKS the rows into bytes and codes those as an 8-bit plane (§2.6.1a). Lossy is refused for the first and supported for the second, where the low-bit objection does not apply | | Indexed color, 8 bpp, 1 channel (mode 2) | Yes at quality 0 — indices and palette, bit-exact in both directions. Lossy refused: quantizing an index yields an arbitrary colour rather than a near one (253 of 256 wrong at quality 4). The palette is returned verbatim and never applied, which is what the reference does | | CMYK (mode 4) | Yes, qualities 0–31 — bit-exact both directions | | RGB48, CMYK64, RGB12, RGB16 (modes 11, 13, 19, 20) | Yes, qualities 0–31 — bit-exact at quality 0 both directions, and both decoders agree on 240 / 240 lossy files written by each side — 60 per format per side; the harness's 420 is its seven-format total, not these four | | L*a*b at 24 and 48 bpp (modes 9 and 12) | Yes, qualities 0–31 — they apply no color transform; 60 / 60 bit-exact at quality 0 in each direction and 240 / 240 lossy agreement. The NAMES are inference; the behaviour is measured | | Grayscale at 16 and 31 used bits (modes 10 and 18 — bpp 16, and bpp 32 with 31 of its bits used) | Yes, qualities 0–31 — single-channel, no color transform, uint16 / uint32 LE samples with half 32768 and 2^30. 60 / 60 bit-exact at quality 0 in each direction, 240 / 240 lossy agreement, 400 / 400 coefficient-exact vs the reference encoder. Neither takes the quality-4 K step | | Modes 5 and 6 — untransformed channels, identity unknown | Yes, qualities 0–31, at 8/1, 24/3 and 32/4. n untransformed 8-bit channels — no colour transform, level shift 128, channels in order 0..n-1. 120 / 120 relabelled reference files where both decoders agree, 180 / 180 our-encode → reference-decode bit-exact at quality 0 (plus 18 / 18 stored-raw), 360 / 360 lossy files where both decoders agree, and 320 / 320 coefficient-exact vs the reference encoder at qualities 0–3. WHAT THE CHANNELS REPRESENT IS UNKNOWN — the colour models are called untransformed5 and untransformed6 precisely so that nothing asserts a colour space; see below. Every other geometry is refused | | More than 31 bit-planes in a macroblock (31-bit content of high contrast) | Refused — the plane-count field is 5 bits, and the reference itself writes a file its own decoder misreads there. The boundary is measured: 31-bit samples encode their FULL range when the content is smooth, but cap at about 2^30 at maximum contrast, because a full-swing alternating signal's highpass needs one plane more than the field allows | | ROI (region-of-interest) coding, container version 0x7E | Yes, both directions. Decoding: 1728 / 1728 files where our decode equals the reference decoder's own output sample for sample, across nine pixel formats, nine geometries, qualities 0, 4 and 7 and one to four levels. Encoding (encode(image, { roi: true })): the same 1728 cases coefficient-exact against the reference encoder's own ROI files, and 1728 / 1728 where the reference decoder reads our ROI file as it reads our ordinary one. ROI changes only how a level's coefficients are partitioned into segments; the transform, quantizer, scan order and payload format are untouched. Writing one is refused for a geometry that clamps to zero pyramid levels (min(w, h) < 10), where the reference codec segfaults on its own files | | Progressive PARTIAL decoding by LEVEL (decode(buf, { maxLevel: k })) | Yes. Consumes only level-directory entries 0..k and returns a reduced image of levelDimensions(w, h, nLevels-1-k) — entry 0 alone is the coarsest thumbnail. A buffer truncated after entry k's bytes is accepted, so a client that fetched a byte prefix can decode what it has. 576 / 576 reference-written prefixes exact, 216 / 216 truncated buffers, across ROI and plain, qualities 0/4/7, 1/3/4 channels and four geometries including non-power-of-two. Read the limit honestly: this is verified by reduction to the already reference-verified full decode, not against a reference reduced decode — libpgf has no level-selective decode mode and refuses truncated files, so no such comparison exists. At quality ≥ 4 with a subsampling format the chroma is upsampled at reduced scale, which nothing can check; those results carry reducedScaleChroma: 'unverified' | | Progressive partial decoding by BLOCK (decode a spatial region of an ROI file) | No, and deliberately so — measured, not merely unoffered. An ROI block is independently entropy-decodable but not independently reconstructable: the inverse lifting crosses block seams, and reading only the overlapping blocks is wrong by up to 34 levels of 255 on ~4% of the region. Exactness needs a halo of one block at every level-directory entry — the minimal halo was 0 or 1 on all 1197 measured cases, never more. It is not shipped because the payoff is absent at the depths real files use: at three levels or fewer an exact region decode reads 100% of the file, and an ROI file is larger than its plain twin. Characterised in docs/SPEC-DIGEST.md §7.5.121 for anyone revisiting it | | Quality 32 and above | Not expressible. The field is a UINT8, but the reference clamps any request above 31 to 31, so 31 is the ceiling and there is nothing beyond it to support |

Container version 0x06 is fully supported. It was refused for a long time, on evidence that this implementation decoded those files incorrectly — at quality 0, where nothing is a free choice, the HL and LH bands came out wrong. Two distinct causes were behind it, and both are now measured:

• 0x06 interleaves HL and LH, running one 4x4 block scan over the rows the two bands share and then appending the taller band's leftover row row-major, instead of scanning each band separately at 8x8 (§4.3).
• 0x06 codes 1-bpp images as PACKED BYTES — it packs the bilevel rows into bytes and codes those as an ordinary 8-bit plane, so a decoder unpacks the reconstructed samples rather than taking each sample's low bit, and the quantizer is grayscale's non-stepping one (§2.6.1a). This is why lossy 1-bpp, refused under every other container, is decodable here.

All ten 0x06 sample files now decode byte-identically to the reference — grayscale, RGB, lena, the 393x501 RGBA pair and both bitmaps, lossless and lossy alike. Nothing in the corpus is refused.

readHeader() parses all of these either way. Versions 0x36 and 0x76 — every digiKam thumbnail, and everything this package encodes — are verified exact and were never affected. See docs/SPEC-DIGEST.md §7.5.124 and §7.5.125, and tools/v0x06-band-check.js for the live measurement.

Grayscale-8's lossy shift was wrong until modes 10 and 18 were measured, and the story is a good illustration of why per-format facts have to be measured per format. The quantization constant K takes a step at quality 4 for the multi-channel 8-bit formats and not for RGB12/RGB16, and grayscale-8 had been set to match the former — it had never been checked, because no available encoder writes mode 1 and the 8-bit lossy conformance figures are all RGBA files.

The two new single-channel formats do NOT take the step, which made mode 1 worth re-examining. It is measurable through the reference decoder, since the shift sits on both sides of a file: write the same image under each candidate rule and see which one libpgf reads back correctly. With RGB and gray16 as controls answering in opposite directions, the answer is unambiguous — mode 1 does not take the step, 24 / 24 files bit-identical under the corrected rule and 0 / 24 under the old one. Under the old rule this package's lossy grayscale files decoded, in libpgf, about 31 grey levels off everywhere. Lossless grayscale was never affected, which is why the 48 / 48 verification below stood while this did not.

How grayscale came to be verified is worth a note, because it was briefly unverified and the fix is instructive. Its original evidence came from the two grayscale sample files — which are 0x06, at the time known to decode incorrectly, so that evidence had to be withdrawn. (It has since been RESTORED by measurement: with the 0x06 interleave implemented, both grayscale samples decode byte-exactly. The independent chain below was built while that was not yet true, and it still stands on its own.) No encoder accepted grayscale input, so no file with known contents could be built to test against.

Adding grayscale encoding supplied one. The chain closes without circularity: we write a grayscale file from a known image, libpgf's decoder reads it back to that image exactly, and our decoder reads it to the same pixels. Two independent decoders agreeing on a file whose contents are known by construction is real verification. 48 / 48 bit-exact, over 16 geometries — 1×1, 2×3, 9×4 and 4×16 among them — × 3 level counts. See docs/SPEC-DIGEST.md §7.5.90 and §7.5.92.

Every pixel format is verified the same way, against libpgf used as a black box. The seven formats libpgf's own wrapper can encode — RGB, RGBA, CMYK, RGB48, CMYK64, RGB12 and RGB16 — are checked in both directions by tools/formats-check.js: 210 / 210 reference-encode → our-decode and 210 / 210 our-encode → reference-decode, both bit-exact at quality 0 (30 cases per format), plus 420 / 420 lossy agreement on the reference's own files and 420 / 420 on ours — 60 per format per side. Expected refusals now score 0 / 0: all seven are supported at every quality the harness offers them, so there is nothing left in it to refuse, and the row is kept rather than deleted so that a reader can see the count went to zero by widening support.

Bitmap and indexed now have both directions, and for a while had only one. The wrapper tool exposed no encode mode for them, so the evidence was our file read back by libpgf — 54 / 54 for bitmap, 90 / 90 wavelet-coded plus 30 / 30 stored-raw for indexed (§7.5.87–7.5.93). That is the direction that matters, but it cannot catch a misconception shared by both sides of one implementation. A rebuilt wrapper now EXPOSES seven encode modes the library always accepted and the CLI never named, and the missing direction runs clean: 132 / 132 in tools/reference-encode-check.js, including the first reference-written INDEXED file this package has read — indices exact, palette verbatim. §7.5.132.

The two L*a*b modes have both directions back, by RELABELLING. libpgf writes no mode-9 or mode-12 file, but a file it wrote in mode 3 or 11 can have its mode byte changed and be handed back to its own decoder — which is what settles the interpretation, with no dependence on our encoder. Both modes turn out to apply no color transform at all: the three planes are the three channels, level-shifted, in caller order 0, 1, 2. 60 / 60 relabelled files decode to exactly what libpgf decodes them to, 60 / 60 of our files read back bit-exact, and 240 / 240 lossy files agree between the two decoders. See §7.5.95 and tools/lab-check.js.

Modes 5 and 6 are implemented WITHOUT being identified, and that distinction is the whole point of the entry. Their coding is measured — n untransformed 8-bit channels for n in 1, 3 or 4, no colour transform, level shift 128, channels in order 0..n-1 — by the same relabelling route, and by our-encode → reference-decode at 8 bpp / 1 channel, where relabelling is impossible because libpgf writes no 8-bit single-channel format. What is not measured is what the channels mean, so the colour models are named untransformed5 and untransformed6 after the mode bytes and assert nothing. Three independent grounds say no measurement could settle it: the two mode bytes are byte-for-byte indistinguishable from each other; at one channel they coincide with mode 1 and at quality 0 with mode 9; and Adobe's list — the only candidate source of names — puts Multichannel at 7 and Lab at 9, values this reference build refuses while recognising 5 and 6.

Two things about them were genuinely open and are now measured. They do not subsample and do not take the quality-4 K step, unlike mode 9, which does both — so they are byte-identical to mode 9 at quality 0 and differ from it above it (one 32×32 file read by libpgf under each label: 3072 / 3072 equal at quality 0, 22 / 3072 at quality 4). And of the geometries libpgf accepts them at — which is all of them — only three reconstruct coherently. The four wide containers put n 8-bit channels in the front of each ceil(bpp/8)-byte pixel slot, zero the rest and clamp a 16-bit plane to 0..255, which is an 8-bit format in an oversized buffer rather than a wide one; the two packed geometries have no reading that scores above noise. All of those are refused. See tools/untransformed-check.js.

Grayscale at 16 and 31 bits needs no such trick — a rebuilt wrapper encodes and decodes both, so both directions are direct. 60 / 60 reference-encode → our-decode and 60 / 60 our-encode → reference-decode, each bit-exact at quality 0 and each checked against the source samples as well as against libpgf's decode; 120 / 120 lossy files agree between the two decoders in each direction; and 400 / 400 quantized coefficients match the reference encoder slot for slot across qualities 0–9. See tools/gray-check.js, which also prints the shift-bias table that measures the K step and the plane-count ceiling both encoders run into.

Indexed color is supported at quality 0, and returns indices. It is not refused any more, and the reason is a measurement that reframed the problem: libpgf does not map indices to colors at all — its output for a mode-2 file is one byte per pixel, the raw index, and applying the palette is the caller's job. So decode() hands back the indices plus the color table's raw 1024 bytes (or palette: null when the file carries none, which libpgf also accepts). No color conversion is invented, because there is none to invent.

The palette's entry ORDER is left to the caller for the same reason. PGF_Details p.2 gives each 4-byte entry as (blue, green, red, alpha-or-unused) and libpgf's buffers are BGR(A) generally, but that cannot be measured with anything available. A palette-mapping mode does exist in the wrapper tool used here, and testing it sharpened the reason rather than removing it: it copies each 4-byte entry into the pixel verbatim, so whatever the output bytes mean is exactly what the table bytes meant, and the ambiguity is relocated rather than resolved. Returning raw bytes cannot bake in an unverified ordering; returning RGBA would. See §7.5.93 and §7.5.133.

Indexed is refused above quality 0, and that refusal is not conservatism: a quantized index is a meaningless quantity, and a wrong index is an arbitrarily wrong color rather than a nearby one. At quality 4, 253 of 256 indices come back wrong. One narrower refusal also survives — a color table on a file whose mode byte is not 2 is still rejected, because that file contradicts itself and nothing measured says which field to believe.

Every quality 0–31 is now measured. The per-band shift is max(0, K + hp - level) with K = q-1 up to quality 3 and K = q-2 from quality 4 on; the step at 4 is where chroma subsampling begins, taking the place of one shift. Qualities 0–9 were pinned byte-exact against a reference decoder, and 10–18 by solving for the shift from the reference encoder's own quantized coefficients — 54 / 54 each way.

Qualities 19–31 were then measured by widening the samples rather than the image. The old bound — a coefficient survives at quality q only if the image is at least 5 * 2^(q-10) pixels on its short side — applies to 8-bit content; a deep LL coefficient is bounded by the sample range, so 31-bit formats carry enough magnitude to survive a shift near 30 on a small probe. 31 is the format's ceiling, the reference clamping anything higher to 31, so the supported range is now the whole expressible one.

Fidelity at quality 4

digiKam thumbnails are quality 4, which is lossy. Channels 1–3 (U, V and alpha) are coded at half resolution in both dimensions, so chroma and alpha detail finer than 2×2 is discarded by the encoder and cannot be recovered by any decoder.

On photographic content — what thumbnails contain — this is visually near-lossless. On synthetic high-frequency chroma (random color noise) the error is large and irreducible. The decoder was checked against the information bound there and sits exactly on it, not above it: a modulo-pattern alpha channel decodes with mean error 57.8, and a 2×2 downsample-then-upsample of the same input gives mean error 57.8 as well. See docs/SPEC-DIGEST.md §7.5.61.

That bound was computed when the encoder's downsampling filter was still unknown, so it stood as an idealization. The filter has since been measured — a 2×2 box average rounded down (§7.5.79) — so the loss it describes is the loss a real encoder inflicts, not a hypothetical one.

Restoring those half-resolution channels needs an interpolation the bitstream does not specify, so it is a caller choice:

decode(buffer, { upsample: 'nearest' })    // default — matches libpgf exactly
decode(buffer, { upsample: 'bilinear' })   // closer to the original image

nearest is the default because it is measured: with it, this decoder reproduces a reference decoder's output exactly on every file it accepts.

bilinear remains available and is not merely a legacy option — it produces a closer reconstruction of the original image, since it interpolates rather than replicating. Conformance and reconstruction quality genuinely diverge here. Choose nearest to match libpgf, bilinear for the better picture.

Testing

npm test

617 tests on the Node built-in runner. No dependencies to install.

Round-trip verification

npm run roundtrip

Builds images, encodes them with an external PGF encoder used as a black box only, decodes them here, and compares pixels. This needs an encoder binary on your machine, so it is not part of npm test.

Fuzzing

The decoder is handed whatever a cataloging application finds on disk, so a malformed file must be an error, never a crash. The property under test:

For any byte sequence, decode() either throws a typed PgfError with a stable code, or returns a complete width * height * 4 buffer. Never a raw TypeError/RangeError from deep inside, never a short buffer, never an unbounded allocation or hang.

npm run fuzz

That is 50,000 iterations, which is the figure the Status table reports: what a reader can reproduce with the shipped script, not a longer run taken on trust. Push it as far as you like:

node tools/fuzz.js --iterations 200000 --seed 42

The PRNG is seeded, so any failure reproduces from its (seed, iteration) pair. A small fixed budget runs as part of npm test for regression cover.

The fuzzer reports which code path each input reached, and the test asserts that the share of inputs producing actual pixels stays above a floor — otherwise a fuzzer that merely bounced off the magic-number check would report zero crashes while proving nothing. Roughly a quarter of inputs reach the entropy decoder.

Encoder verification

npm run encode-check

Decoding our own encoder's output proves only self-consistency: a misreading of the format would be symmetric between the two halves and invisible. The checks that matter are the ones that are not circular.

Conformance is established. Files written here are read back by libpgf's own decoder, byte for byte:

| Evidence | Result | |---|---| | libpgf decodes our output to the source pixels | 360 / 360 bit-exact | | — grayscale / RGB / RGBA | 120 / 120 each | | Self round-trip bit-exact, this harness's own sweep | 50 / 50 | | Container fields agree with the reference encoder's | 50 / 50 | | Compressed size vs libpgf, same coding choices | mean 1.005×, worst 4.3% | | Lossy q4–q9: quantized coefficients vs the reference encoder | 324 / 324 exact | | Lossy q4–q9: pixels through the reference decoder | 324 / 324 identical | | Qualities 0–31, quantized coefficients vs the reference encoder | 960 / 960 exact | | Grayscale, our encode → libpgf decode, 16 geometries × 3 depths | 48 / 48 bit-exact |

The two rows this script itself produces cover its own grid: 5 patterns (gradient, plasma, rings, flat and a modulo alpha channel) × 5 geometries (16×16, 64×64, 100×60, 128×128 and 256×171, the last with an odd height) × 1 and 3 pyramid levels — 50 cases, each encoded at the level count the reference itself chose, since libpgf clamps depth by image size and comparing across different depths would compare different things. There is no 8×8 case and no random-noise pattern in it. The other rows come from other harnesses — the quality sweep from tools/pgfa-quality-check.js, the grayscale row from tools/gray-check.js. The two lossy rows are a separate sweep: 9 geometries, odd widths and heights included, × 2 patterns × qualities 4–9 × 1–3 levels. Lossy output cannot be compared to the source, so the comparison is against the reference encoder's file — first by entropy-decoding both and diffing quantized coefficients slot by slot, then by running both through the reference decoder and diffing pixels.

The size comparison is a useful secondary signal: an entropy coder's output size is a sensitive function of every coding decision, so tracking a reference implementation to within 1% means substantially the same decisions are being made. Where sizes do diverge it is explained — this encoder always writes RLR significance, and where libpgf chooses a RAW plane instead our files run 4–6% larger. That is policy, not error.

This matters as a cautionary note as well. For a long stretch the encoder passed self round-trips 40/40 while libpgf could read only 8 of 40 of its files: our encoder and decoder agreed with each other and both disagreed with the reference. Self-consistency is worth very little on its own.

Test corpus

Three directories, all gitignored and never committed, and the split is by what the files ARE rather than by what they test:

| directory | contents | why it is not committed | |---|---|---| | corpus/ | the 2015 libPGF sample pack — ten 0x06 files | third-party sample pack, not redistributable | | corpus-local/ | eleven digiKam thumbnails, 0x76 | derived from personal photographs | | corpus-extra/ | one 0x36 file from ExifTool's test data | third-party test data |

corpus-local/ is named for what it holds. A thumbnail cache is generated from whatever images its owner has, so it is personal data that happens to be useful as test input — and the path should say so without anyone having to read .gitignore to find out. Generate your own locally; see tools/analysis/prep-digikam.js.

The committed fixtures in test/fixtures/ are small synthetic images generated for this project (gradients, rings, a dense ±1 construction). They contain no personal data, and between them they exercise both sign-plane codings.

Documents

| File | Contents | |---|---| | docs/PGF-FORMAT.md | A complete, language-agnostic specification for decoding AND encoding PGF. Everything needed to write a codec in any language, including the entropy coder the published papers omit, the encoder-side quantizer and depth clamp, and ROI writing. It claims that a conformant decoder and encoder can both be built from it alone — not that an encoder built from it emits bytes identical to any particular reference build, since several choices are genuinely free. MIT licensed. | | docs/pgf-bitstream.html | The same specification as a standalone, offline HTML page — open it in any browser, no server or network needed. Generated from docs/PGF-FORMAT.md by npm run render-spec; never edited by hand, and the test suite fails if the two disagree. There is no hosted copy: this file, in this repository, is the only edition. | | CLEANROOM.md | Clean-room methodology, source rules, authorship disclosure, and an unsigned attestation block. Read before relying on the provenance claim. | | docs/V0X06-PROTOCOL.md | How the one frozen investigation — the container 0x06 band defect — may be resumed without the frozen party steering the result. Published before the work ran, so the design could not be retrofitted to an outcome. Run, and it returned NO_SURVIVOR — the rule was in neither committed candidate space (§7.5.123). That null was CORRECT: the answer moves coefficients between HL and LH, which neither space could express. It was found afterwards by a third party given only probes and no history, and reproduced here (§7.5.124). | | docs/SPEC-DIGEST.md | The substantive record: every rule, the measurement that established it, and the hypotheses that were retracted. | | docs/SOURCE-LOG.md | Append-only log of every source consulted, with dates. | | docs/BITSTREAM-EXPERIMENT.md | Working notes from the bitstream measurement sessions. |

The two published PGF papers are not redistributed here — they are third-party copyrighted works. CLEANROOM.md §2 says how to obtain them.

Contributing

The one rule that is not negotiable: do not fill an undetermined value with a plausible guess, even if it appears to work. Add a measurement to docs/SPEC-DIGEST.md that establishes it, or leave the typed error in place. A guess that happens to produce correct pixels is the exact failure mode this project is built to avoid, for the reasons set out in CLEANROOM.md.

Contributions must also respect the source rules in CLEANROOM.md §3. If you have read libpgf's source, please do not contribute to the decode path.