@moniav/hebrew-pdf

A standalone Node.js/TypeScript library for ingesting heterogeneous Hebrew PDFs into logical-order text, clean Markdown, structured tables, and a page-level audit report.

It is designed as shared infrastructure for Tessera, BursAIQ, TASE_data, LegalAnalytics, and future services. It has no dependency on any of those applications.

Coding agents (Claude Code, Codex) integrating this library into a consumer project should read AGENT_GUIDE.md, the condensed decision-oriented reference that ships inside the package at node_modules/@moniav/hebrew-pdf/AGENT_GUIDE.md.

Design promise

The library does not claim that one parser can correctly process every PDF. Instead, every page follows an explicit route and must pass a Hebrew quality gate. A page is either:

1. accepted from the native PDF layer;
2. recovered by an independent text extractor;
3. recovered by an explicitly enabled structured extractor;
4. recovered by OCR;
5. or marked review-required without emitting convincing but untrusted text into the canonical export. Every review-required page carries a quarantined reviewPreview in the structured page model, built from the best rejected candidate, so a reviewer can compare it with the source.

It never reverses whole Hebrew strings.

Validated results

Version 0.2.0 was validated end to end by independent review of the extraction against rendered page images.

• Bank of Israel credit-risk directive (62 pages of dense regulatory Hebrew): 52 pages accepted, 87.8% of the Hebrew characters the text layer holds, with correct clause numbering (65ב.), bullets, headings, linked footnotes, and formulas decoded to real operators. The 10 withheld pages are ruled tables and diagrams that correctly fail closed, each with a quarantined preview.
• General table-of-contents document (4 pages): 4 accepted, 100% coverage; with structured recovery enabled the contents emit as typed tables with correct row pairing.
• An unseen mixed Hebrew/English legal business document extracted cleanly on the first run (2/2 pages, exit 0), confirming the heuristics are not fitted to the fixtures.

Install

pnpm add @moniav/hebrew-pdf

Runtime: Node.js 22 or newer. The default local adapters also expect Poppler (pdftotext, pdftoppm). OCR requires Tesseract with heb+eng language data. Optional structured recovery uses Python with pymupdf4llm installed and is enabled explicitly with --structured-recovery pymupdf4llm or the matching library option.

pnpm install
python -m pip install pymupdf4llm

Install Poppler and Tesseract through the operating system package manager and ensure their executables are on PATH. On Windows, pdftotext and pdftoppm are also discovered in the WinGet Poppler package, and Tesseract in its standard %ProgramFiles%\Tesseract-OCR location, so no PATH editing is required for those installs. When Hebrew language data cannot be placed next to Tesseract, point the standard TESSDATA_PREFIX environment variable at a directory containing heb.traineddata and eng.traineddata. A missing binary produces an actionable error naming it and how to install it, and the report's engineHealth.ocr key shows whether the OCR route was available, not_needed, disabled, or failed (with ocrError). PyMuPDF4LLM is optional; leave structured recovery off when Python is unavailable.

The package is consumable today from a registry, a packed tarball (npm pack), a git dependency, or a pnpm workspace; the packed artifact has been verified end-to-end from a clean consumer project.

Library API

import { createHebrewPdf } from "@moniav/hebrew-pdf";

const hebrewPdf = createHebrewPdf({
  ocr: "auto",
  structuredRecovery: "pymupdf4llm",
  maxPages: 500,
});

const result = await hebrewPdf.ingest("./document.pdf");

console.log(result.markdown);          // display/export artifact
console.log(result.text);              // logical text with \f page separators
console.log(result.document.pages);    // canonical structured page model
console.log(result.report.needsReview);

Buffers are supported without the caller managing a temporary file:

const result = await hebrewPdf.ingest({
  data: uploadedBuffer,
  filename: "דוח-שנתי.pdf",
});

There is also a one-shot helper:

import { ingestHebrewPdf } from "@moniav/hebrew-pdf";

const result = await ingestHebrewPdf(buffer, { ocr: "off" });

Errors

ingest() rejects with a typed error for every fatal condition. Each error extends HebrewPdfError and carries a stable, machine-readable code. Branch on code, not on message text, since wording may change between releases.

| Code | Class | Meaning | |---|---|---| | invalid_pdf | InvalidPdfError | The input is not a parseable PDF: a missing header or a native parser failure. The original native error is preserved as cause. | | encrypted_pdf | EncryptedPdfError | The PDF appears to be password-protected/encrypted. | | pdf_too_large | PdfTooLargeError | The input exceeds maxBytes. A string-path input is size-checked before the full file is read into memory. | | too_many_pages | TooManyPagesError | The PDF has more pages than the configured maxPages. | | empty_input | EmptyInputError | The input buffer is empty. | | zero_pages | ZeroPagesError | The PDF parsed but reported zero pages, and does not look encrypted. | | invalid_options | InvalidOptionsError | A caller-supplied option failed validation, for example ocr: "yes-please" or maxPages: 0. | | aborted | AbortedError | Ingestion was cancelled through signal. |

import { HebrewPdfError, createHebrewPdf } from "@moniav/hebrew-pdf";

try {
  await createHebrewPdf().ingest("./document.pdf");
} catch (error) {
  if (error instanceof HebrewPdfError) {
    console.error(error.code, error.message);
  }
  throw error;
}

Every code is exported as the HebrewPdfErrorCode union type. HEBREW_PDF_ERROR_CODES lists them all at runtime.

Cancellation

Pass an AbortSignal to cancel ingestion. The signal is checked before work starts and again between pages; a page already mid-extraction still finishes before the next check. The bundled Poppler fallback adapter kills its subprocess as soon as the signal fires, instead of waiting out the full command timeout.

const controller = new AbortController();
const resultPromise = createHebrewPdf({ signal: controller.signal }).ingest("./document.pdf");
setTimeout(() => controller.abort(), 5000);
await resultPromise; // rejects with AbortedError if the timeout wins the race

Canonical output

Markdown is not the internal data contract. Each result includes a serializable HebrewPdfDocument:

interface HebrewPdfDocument {
  schemaVersion: 1;
  language: "he";
  direction: "rtl";
  source: { filename: string; byteLength: number; sha256: string };
  pageCount: number;
  pages: Array<{
    number: number;
    route: PageRoute;
    markdown: string;
    text: string;
    blocks: Array<Heading | Paragraph | List | Table | Review>;
    rawText: string;
    edits: NormalizationEdit[];
    editMap: NormalizationMapStatus;
    tableDetected: boolean;
    needsReview: boolean;
    quality: QualityResult;
    reviewPreview?: { markdown: string; text: string; blocks: DocumentBlock[] };
  }>;
}

reviewPreview exists only in this document model. The canonical Markdown export intentionally carries just a review marker comment for a rejected page, and the audit report carries the rejection reasons; the quarantined candidate text never leaks into either.

Structured table blocks contain headers, rows, and the original Markdown. This lets an application render a table, index individual cells, or derive plain text without reparsing the complete document. Pages also expose normalization telemetry so a reviewer can see exactly what changed: nfkcChanges, rtlPunctuationRepairs, acronymRepairs, latinBoundarySpaces, directionalControlsRemoved, splitWordRejoins (extractor mid-word breaks rejoined with page-level evidence), and symbolGlyphsMapped (Symbol-font formula glyphs decoded to Unicode operators).

Raw text and the edit list

Each page carries the text its route produced before any repair ran, as rawText, and an ordered non-overlapping list of the replacements that turn it into the page's Markdown, as edits. This is what makes exact citation against the source possible. On a real regulatory page most lines carry a directional control that normalization removes, so a quotation of more than one line taken from the display text is not a substring of the raw text.

Verify the map on ingest instead of trusting it. A page whose map does not replay is a library regression, and catching it here is the difference between a loud failure now and a wrong citation weeks later.

import { applyEdits, PACKAGE_VERSION, verifyEdits } from "@moniav/hebrew-pdf";

for (const page of result.document.pages) {
  const display = page.editMap.target === "markdown" ? page.markdown : page.reviewPreview?.markdown ?? "";
  const check = verifyEdits(page.rawText, page.edits, display);
  if (!check.ok) throw new Error(`page ${page.number}: map from hebrew-pdf ${PACKAGE_VERSION} did not replay at ${check.firstMismatchAt}: ${check.reason}`);
}

applyEdits(rawText, edits) replays a map and returns the display text. It throws RangeError on a list that is out of order, overlapping, or out of bounds, so verifyEdits is the call to use when a verdict is wanted rather than an exception.

Each edit records the raw range it replaces, the replacement, the display range that replacement occupies, and the repair that made it. The kinds are nfkc, bidi_control_removed, rtl_punctuation_repair, acronym_repair, latin_boundary_space, split_word_rejoin, symbol_glyph_mapped, line_ending, whitespace, and layout_reconstruction for the plain-text route's rebuild of the page's line layout. Offsets are UTF-16 code units and both ranges are half-open.

editMap says where the map stands. target is the text the edits reconstruct: markdown for an accepted page, reviewPreview for a refused one, whose canonical Markdown is only a review marker. complete is false when the map must not be used to project a citation, which is the case for every refused page (gap: "page_refused"), a page no route could read (gap: "no_text"), and a page whose map the library replayed and found wrong (gap: "replay_failed"). origin is engine_output except on the one route that concatenates two engines' output for a recovered table, where it is composed_candidate.

What rawText holds depends on the route: the native extractor's Markdown, the plain-text engine's own page text, the structured engine's Markdown, or an OCR engine's reading of the page. On the plain-text route the rebuild of the page's line layout is recorded as edits like any other stage, so the map reaches that engine's output rather than stopping at the rebuilt text.

For digital Hebrew documents, the native adapter also consumes positioned text and tagged-PDF structure from pdf-inspector. Right-margin regulatory numbers are reconstructed as explicit clause blocks, bold numeric/Hebrew markers become section headings, parenthesized Hebrew markers become ordered list items, and small-font footer notes become footnote blocks. Geometry supplies boundaries only; Poppler remains the independent source for logical wording. When a geometry-confirmed marker does not survive into canonical blocks, the pipeline reports numbered_structure_loss. It fails the page closed only when the marker's text is missing from the page entirely, which is content loss; when the text is present but was not reconstructed as its own block, the page is accepted with the same code as a warning. Single-character markers never fail a page closed, because the geometry detector reports a stray margin letter as a list item and a lone letter or digit is present in almost any Hebrew page.

Extraction routes

flowchart TD
    A[PDF page] --> B[pdf-inspector]
    B -->|image-only page| S{Structured recovery enabled?}
    B --> C{Native quality gate}
    C -->|pass| D[Canonical blocks]
    C -->|fail| E[pdftotext fallback]
    E --> G{Fallback gate incl. corroborated-table check}
    G -->|pass| D
    G -->|fail| S
    S -->|yes| I[PyMuPDF4LLM structured recovery]
    I --> J{Quality, word geometry, and 85% cross-engine agreement}
    J -->|pass| D
    S -->|no| O{OCR enabled?}
    J -->|fail| O
    O -->|yes| F[pdftoppm + Tesseract heb+eng, or custom OCR command]
    O -->|no| H[Review marker + quarantined reviewPreview]
    F --> K{OCR quality gate}
    K -->|pass| D
    K -->|fail| H

Every gate applies the same Hebrew quality assessment; a page only reaches review-required after each enabled route has been tried and rejected, and the best rejected candidate is kept as the page's reviewPreview.

The bundled native adapter pins @firecrawl/pdf-inspector 1.14.0. It provides PDF classification, reading order, multi-column detection, and Markdown tables. pdftotext provides an independent fallback. Tesseract is the default OCR route for plain scanned text. Pages the native adapter identifies as image-only skip the text fallback and continue to the enabled structured/OCR recovery stages.

Replaceable adapters

All engines implement PipelineDependencies:

interface PipelineDependencies {
  inspect(buffer: Buffer): Promise<NativeDocument>;
  extractFallbackPages(
    path: string,
    pageCount: number,
    options?: { timeoutMs?: number; signal?: AbortSignal },
  ): Promise<string[]>;
  extractStructuredPage?(input: StructuredPageInput): Promise<{
    markdown: string;
    structured: true;
    engine: string;
  }>;
  extractOcrPage(input: OcrPageInput): Promise<{
    markdown: string;
    structured: boolean;
    engine: string;
  }>;
}

Inject adapters at construction:

const hebrewPdf = createHebrewPdf({
  dependencies: {
    inspect: myNativeExtractor,
    extractFallbackPages: myFallback,
    extractStructuredPage: myLayoutAwareExtractor,
    extractOcrPage: myLayoutAwareOcr,
  },
});

This is the extension point for cloud OCR, a vision model, GPU table recognition, or a sandboxed extraction worker. Bundled adapters are also exported from @moniav/hebrew-pdf/adapters.

Tables and scanned documents

Native Markdown tables are validated for a real header/separator and consistent cell width. Ordinary Hebrew prose containing | is not treated as a table.

Plain-text fallback and plain Tesseract cannot reconstruct table geometry. A native table suspicion is not trusted on its own: the detector mistakes right-aligned RTL prose for a grid often enough that acting on every suspicion would delete most of a clean document. The page fails closed when a second, independent engine corroborates the suspicion, meaning the native Markdown itself holds a compact multi-column table, the Poppler -layout text holds at least three consecutive column-aligned rows of short cells, or word geometry confirms a table during structured recovery. An uncorroborated suspicion is reported as a plain_text_cannot_recover_tables warning on the accepted page so a reviewer can still check it. The report distinguishes the two views: pagesWithTables lists accepted pages whose output holds a verified Markdown table, while pagesWithSuspectedTables carries the native layout detector's unfiltered suspicions, including rejected pages. Enable the optional PyMuPDF4LLM structured recovery, configure another layout-aware adapter, or use the CLI's --ocr-command contract. Structured output is checked against independent Poppler text. Both word precision and recall must reach 85%; missing reference text, a missing detected table, or lower agreement fails closed. Reports expose agreement and engine health. The OCR command receives the absolute PDF path and a 1-based page number and prints Markdown.

The bundled structured adapter captures PyMuPDF word and line geometry before importing PyMuPDF4LLM, then reconstructs RTL cells in logical word order. Borderless contents/index grids are reduced to typed תיאור / מספר tables; text-aligned prose is not promoted to a table. Numeric columns are rendered as isolated LTR values while Hebrew headers and labels remain RTL. MuPDF parser diagnostics are routed to stderr and filtered, so engine messages never appear as document content.

For table-of-contents and other column-layout documents, enable structured recovery. Without it, such pages are still extracted with complete text in logical order, but as prose paragraphs rather than typed table rows; with it, the same pages route through geometry and emit real Markdown tables with correct row pairing.

CLI

The CLI is an optional thin wrapper around the same public API. Once the package is installed as a dependency, the hebrew-pdf binary is available directly:

npx hebrew-pdf document.pdf --out document.md --report document.report.json

When working inside this repository, the same CLI runs from source:

pnpm install
pnpm build
pnpm ingest document.pdf \
  --structured-recovery pymupdf4llm \
  --out document.md \
  --report document.report.json \
  --json document.structured.json

Useful options:

• --ocr auto|off|force controls whether unresolved pages enter OCR.
• --structured-recovery off|pymupdf4llm enables optional layout-aware recovery.
• --ocr-command EXECUTABLE supplies a custom page-level Markdown OCR program.
• --max-pages NUMBER rejects oversized documents before extraction.

Exit code 0 means every page passed; 2 means artifacts were written but review is required; 1 means a fatal processing error.

Human review demo

Run the included local review studio:

pnpm demo

Then open http://127.0.0.1:4173. The demo supports PDF upload, document and page-level statistics, synchronized source/result page navigation, rendered and raw extraction views, engine health (Poppler, structured, and OCR), cross-engine agreement, risk-sorted pages, issue details, next-flagged navigation, approve/flag decisions, reviewer notes, unresolved-page filtering, and review JSON export. Uploaded PDFs and ingestion results are retained only in server memory; the six newest sessions are kept until the demo process exits.

The rendered view mirrors the source page's RTL layout: clause numbers anchor at the right margin with the clause text flowing left of them, Hebrew-letter and numeric list markers render on the right, tables render RTL with numeric columns as isolated LTR cells, and Latin runs inside Hebrew text are bidi-isolated so mixed sentences read exactly as the original does. Rejected pages show their quarantined reviewPreview behind an explicit untrusted-content banner so a reviewer can compare it with the source before approving anything.

Hebrew quality policy

Detection runs before destructive cleanup. Signals include:

• minimum readable text and expected Hebrew ratio;
• visual-order (reversed) Hebrew, detected structurally rather than by vocabulary: Hebrew final forms (ך ם ן ף ץ) only end words and their medial counterparts only occur mid-word, so words that begin with a final form or end with a medial one indicate mirrored text in any Hebrew, with a closed anchor list kept as an additional tripwire for short fragments;
• broken CID/font mappings and replacement characters;
• private-use glyphs that survived Symbol-font decoding (unmapped_private_use_glyphs), so a formula that would render as empty boxes can no longer be accepted silently;
• Hebrew presentation forms and fragmented letters;
• bidi control characters;
• malformed Markdown table widths;
• structured/Poppler disagreement below 85% precision or recall;
• missing structured tables where native layout detection found one;
• fragmented or paragraph-sized headings;
• unbalanced (), [], and {} punctuation;
• explicit image/picture omission markers;
• ambiguous Hebrew/number boundaries, preserved and retained as warnings;
• native extractor OCR recommendations.
• positioned marker loss between geometry and canonical blocks.

Normalization happens only after acceptance or inside a quarantined review preview:

• NFKC normalization and bidi-control removal;
• punctuation displaced by the source's directional embeddings (U+202A..U+202C) is resolved at the embedding edges before the controls are stripped, so marks read on the side a human sees in the original; signs written directly onto a token (AA-, 100%) are preserved;
• bounded mirrored-punctuation repair under a balance invariant: a line whose brackets already nest correctly is never rewritten, and a rewrite must leave the line balanced;
• Hebrew/Latin/digit boundary spacing, with alphabetic clause ordinals (65א) protected;
• Symbol-font private-use glyphs decoded to real Unicode operators and Greek letters;
• extractor mid-word breaks rejoined only when the page itself proves the cut (the orphan tail never stands alone on the page and the joined word appears intact elsewhere on it);
• footnote digits glued to a word become [^N] references when geometry confirmed that footnote's definition on the page.

Hebrew/number adjacency is otherwise preserved. English text, URLs, email addresses, dates, decimal numbers, %, and ₪ are not reversed. Correct input round-trips byte-identical through the whole normalization and block-parsing path, enforced by tests.

Supported PDF classes

Semantic headings include numbered sections, nested subsections, notes (ביאור N), appendices, and common Hebrew section titles instead of trusting visual font size alone.

The routing design covers digital text PDFs, scanned PDFs, mixed text/scan documents, multi-column pages, RTL/LTR mixed text, native tables, and pluggable scanned-table OCR. Malformed, encrypted, corrupt, rotated, custom-font, sparse-table, and misleading text-layer cases must be represented in a regression corpus. See fixtures/README.md.

“All kinds” means the library has a safe route or explicit failure state for each class—not that any engine can guarantee perfect extraction from every possible PDF.

Project checks

pnpm check
pnpm test
pnpm build

Security boundaries

• 100 MiB default input limit and configurable page limit, both validated as typed errors rather than silently dropped;
• 120-second subprocess timeout, 64 MiB stdout limit, and 256 KiB stderr limit;
• shell: false for subprocesses;
• caller-supplied filenames are sanitized to a safe basename before touching disk;
• temporary uploads and OCR images removed on both success and failure, including a failed write;
• cooperative cancellation via AbortSignal, checked before work starts and between pages;
• failed candidate text excluded from final output.

For public uploads, run PDF/OCR tools in isolated workers with CPU, memory, concurrency, malware-scanning, and object-storage controls.

License

MIT. Dependencies and system tools retain their own licences.