pdf-chunk-bbox

RAG over PDFs is easy right up to the moment the answer says "according to your lease, the rent is €1,200" and the reader asks where. Retrieval gave you a chunk of text; the reader wants a marker pen on page 2.

This library is that last mile:

PDF ──extract──▶ text + positioned runs ──chunk──▶ chunks + line rectangles
                                                        │
                        model quotes an excerpt ─────────┴──locate──▶ page + rectangles to draw

Coordinates come out normalised to 0..1 with a top-left origin, so a rectangle can be dropped into an overlay as CSS percentages and stays correct at any zoom.

Install

npm install pdf-chunk-bbox

Node ≥ 20, ESM. The core has no dependencies. The PDF extractor lives behind a separate entry point and needs unpdf as an optional peer — bring your own pdfjs pass instead if you already have one.

Quickstart

import { chunkTextWithBboxes, locateExcerpt } from "pdf-chunk-bbox";
import { extractPdfWithCoords } from "pdf-chunk-bbox/extract";

// 1. Indexing time
const { text, items } = await extractPdfWithCoords(pdfBytes, { maxPages: 200 });
const chunks = chunkTextWithBboxes(text, items);
// store chunk.content (embed it), chunk.pageNumber, chunk.bboxes

// 2. Answer time — the model quoted a passage from a chunk
const found = locateExcerpt(excerpt, chunk.content, chunk.bboxes);
if (found) {
  scrollTo(found.page);
  for (const box of found.bboxes) {
    draw({ left: `${box.x * 100}%`, top: `${box.y * 100}%`,
           width: `${box.w * 100}%`, height: `${box.h * 100}%` });
  }
}

found is null when the excerpt could not be placed with confidence. Show no highlight then: an approximate one teaches the reader that the highlights cannot be trusted, which costs more than the feature is worth.

See it

npx tsx examples/demo.ts && xdg-open examples/demo.html

Renders a two-page document from the library's own coordinates and overlays the highlights it computed for three quotes — one of them whitespace-mangled the way a model mangles it. If the rectangles line up with their sentences, the maths is right; if they drift, you can see it drift.

Why one rectangle per line

A single box around a multi-line passage covers the margins and the lines above and below it. It reads as "somewhere on this page". Per-line rectangles read as a marker pen — so computeChunkBboxes groups runs by baseline and emits one rectangle per line, sorted in reading order.

Runs are grouped when their normalised y differs by less than lineTolerance (default 0.01, about a third of a line at A4/12pt). Raise it for scans and OCR output whose baselines wobble; lower it for dense tables where adjacent rows would otherwise merge into one band.

The invariant everything rests on

chunk.content === text.slice(chunk.charStart, chunk.charEnd)

Bboxes are computed from source offsets; an excerpt is located inside content. If the two drift by one character, the highlight lands on the neighbouring line. So chunks are accumulated as ranges rather than concatenated strings, and trimming adjusts offsets instead of editing text. There is a test for it.

The same care applies to fuzzy matching. Models normalise whitespace when they quote — two spaces become one, a mid-sentence line break disappears — so the excerpt is matched against a normalised chunk. But the highlight needs offsets in the original, and rescaling by the length ratio (normIdx / normLen * originalLen) is off by however much whitespace was collapsed before the match. normaliseWithMap keeps a per-character index map so the mapping back is exact.

Three strategies are tried in order, and found.strategy tells you which one hit — worth logging, since a rising share of anything other than exact means your prompt has stopped asking for verbatim quotes:

| strategy | When | | --- | --- | | exact | The excerpt is a substring of the chunk. The healthy case. | | normalised | Matches once whitespace and case are ignored. | | prefix | Only the opening matches (default: first 40 characters) — the model started quoting and drifted. The highlight covers the excerpt's length, clamped to the chunk. |

Chunking

chunkText(text, options) splits on blank lines, with configurable targetSize (default 1500 chars, ~400 tokens), maxSize (3000) and overlapSize (200). chunkTextWithBboxes(text, items, options) does the same and attaches the rectangles.

Two behaviours worth knowing, because they are not what a naive implementation does:

• No chunk exceeds maxSize. A PDF text layer emits a single newline per line break, so a whole page is usually ONE blank-line-delimited paragraph. A ceiling that only applies when combining paragraphs therefore never applies at all, and a dense page comes out as one chunk several times over the stated limit. Oversized blocks are cut — preferring a line break, then any whitespace, then a hard cut — into pieces small enough that the prepended overlap still fits.
• No duplicate tail chunk. When the last paragraph tips the buffer past targetSize, the buffer is flushed and reset to the overlap; a final unconditional flush would then emit that overlap as a chunk of its own — content already indexed, embedded and paid for twice.

Limits

• Text-layer PDFs only. A scan has no text layer: run OCR first and feed its word boxes in as PdfTextItem[]. Nothing here does OCR.
• Horizontal left-to-right text. Rotated pages, vertical scripts and RTL are untested; the transform matrix is read for position and scale, not orientation.
• Line grouping is baseline proximity, not layout analysis. Two columns at the same height merge into one wide rectangle. If you need column awareness, cluster the items by x before calling computeChunkBboxes.
• prefix matching can under- or overshoot. It extrapolates the end from the excerpt's own length. It is a fallback for a drifting model, not a substitute for asking the model to quote verbatim.
• Run width comes from the PDF. When the text layer omits it, it is estimated from the character count and the rectangle runs slightly wide.
• extractPdfWithCoords has no default page cap. pdfjs text extraction is synchronous and CPU-bound; a thousand-page PDF will block the event loop long enough for a health check to fail. Pass maxPages on a server and read skippedPages.

API

| Export | Purpose | | --- | --- | | chunkText(text, opts?) | Chunks with source offsets. | | chunkTextWithBboxes(text, items, opts?) | Same, plus per-line rectangles and pageNumber. | | computeChunkBboxes(items, start, end, opts?) | Rectangles for an arbitrary character range. | | locateExcerpt(excerpt, content, bboxes, opts?) | { page, bboxes, spansPages, strategy, charStart, charEnd } or null. | | findChunkByExcerpt(chunks, excerpt, opts?) | Which chunk really holds a quote, when the model returns a bogus id. | | normaliseWithMap(s) / toOriginalRange(n, a, b) | Whitespace-insensitive matching that maps back exactly. | | extractPdfWithCoords(bytes, opts?) | (pdf-chunk-bbox/extract) PDF → { text, items, pages, skippedPages }. |

An excerpt crossing a page break returns the page it starts on, plus spansPages listing the rest — rather than silently keeping whichever page held the most lines.

Development

npm install
npm test
npm run typecheck
npm run build

48 tests, no network, no binary fixtures. The extractor is tested against a real parser using a PDF assembled by hand in src/__tests__/pdf-fixture.ts, including an end-to-end case: PDF bytes in, highlight coordinates out.

Provenance

Extracted from the document-indexing layer of a production SaaS that highlights cited passages in legal filings. Four defects surfaced during extraction and are fixed here: the unbounded chunk size and the duplicate tail chunk described above, the whitespace-ratio mapping that drifted a highlight onto the neighbouring line, and a page separator inserted twice, which inflated every offset past the first page break.

License

MIT