@bandf/babel

Document-to-markdown converter for turning common document formats into clean Markdown.

@bandf/babel converts PDF, raster images, EPUB, Office/OpenDocument files, structured data, HTML, plain text, Markdown, and http(s) URLs into normalized Markdown plus source metadata. One function in, clean Markdown out.

Requires Node.js 24 or newer.

import { convert } from '@bandf/babel';

const result = await convert({ filePath: './paper.pdf' });
const markdown = result.markdown;
// → '# Title\n\nbody prose...\n\n## Section\n\n...'

Two entry points

• convert({ filePath, langs }) — the main export. Document file → conversion result with markdown, source fields, MIME type, and parser metadata. Used to bring foreign formats into Markdown. Lazy-loads the format parsers, so their heavy libraries load only when you actually convert a document.
• createBabel(config) — builds a configured converter instance. Use this when you want instance-level defaults for deterministic conversion options.
• @bandf/babel/validate — validate({ markdown }). A markdown string in, cleaned-and-validated markdown out. Uses the shared GFM parser/serializer, but none of the format-specific document extractors. convert() runs every result through it.

import { validate } from '@bandf/babel/validate';
const clean = await validate({ markdown: someUntrustedMarkdown });

Markdown Contract

babel's output contract is clean GitHub Flavored Markdown with useful document structure preserved. Every conversion produces:

• ATX headings (# , ## …) — stable section boundaries for downstream consumers.
• Pipe-format tables — kept together as one block, cells preserved.
• GFM structure — links, images, blockquotes, fenced code, and tables survive when the source format exposes them.
• Normalized prose — homoglyphs transliterated, smart punctuation and invisible characters stripped, no raw HTML tag residue.

A convert() call either returns a result whose markdown field is valid markdown or throws. validate() returns valid markdown or throws. The guarantee is enforced by the parser, conversion, and validation tests in this package.

API

| Export | Returns | Notes | |---|---|---| | convert({ filePath, langs }) (@bandf/babel) | Promise<BabelConversionResult> | Zero-config document → { markdown, sourceUri, sourceType, mimeType, extension, meta }. filePath may be a path or an http(s) URL. Lazy-loads the format parser. langs applies to OCR-backed PDF/image conversion. | | createBabel(config) (@bandf/babel) | { convert, validate, releaseScribe } | Builds a configured converter instance. | | validate({ markdown }) (@bandf/babel/validate) | Promise<string> | Markdown string → cleaned + validated markdown. No parser deps. | | releaseScribe() | Promise<void> | Cancels any active OCR child process. Safe to call after a batch of PDF/image conversions. | | SUPPORTED_EXTENSIONS | string[] | The recognized file extensions. | | SUPPORTED_IMAGE_EXTENSIONS | string[] | OCR-backed image extensions: .png, .gif, .jpg, .jpeg, .webp. |

Supported formats

| Format | Engine | |---|---| | .txt / .text | UTF-8 text cleanup, then markdown-normalized + validated | | .md / .markdown | read as markdown, then markdown-normalized + validated | | .html / .htm | Defuddle article HTML extraction → Turndown/GFM markdown | | .docx | mammoth → semantic HTML → Turndown/GFM markdown | | .csv / .tsv | csv-parse → GFM table | | .json | JSON parse/validate → fenced json block | | .yaml / .yml | YAML parse/validate → fenced yaml block | | .xml | XML parse/validate → fenced xml block | | .xlsx | Open XML workbook parts → one GFM table per sheet | | .ods | OpenDocument spreadsheet content.xml → one GFM table per sheet | | .pptx | Open XML presentation parts → slides, text, notes, and image alt text | | .odt | OpenDocument text content.xml → headings, paragraphs, lists, and tables | | .rtf | local RTF text extraction for common controls and Unicode escapes | | .pdf | scribe.js-ocr (text + OCR), isolated in a child process | | .png / .gif / .jpg / .jpeg / .webp | scribe.js-ocr image OCR, isolated in the same child process path | | .epub | epub2 spine → chapter XHTML → Turndown/GFM markdown | | http(s) URL | SSRF-guarded fetch → content-type dispatch |

Spreadsheet conversion extracts visible values only. It does not evaluate formulas, render charts, or OCR images embedded inside Office/OpenDocument containers. PPTX conversion extracts slide text, speaker notes, and image alt text; embedded media stays out of scope unless it is already exposed as text.

URL conversion is intended for library use in trusted/local jobs. The built-in SSRF guard blocks common private/internal targets, checks DNS results before fetching, validates every redirect before following it, and enforces redirect and response-size limits. Hosted multi-tenant deployments should still add network sandboxing around calls that accept user-provided URLs.

The format-parser libraries install with @bandf/babel, so consumers do not need to manage parser-specific peer dependencies. They are still lazy-loaded: validate() loads only the markdown normalization stack, and convert() loads only the parser required for the document being converted. PDF and image OCR run in a short-lived child process because scribe.js-ocr is not re-entrant.

English OCR works offline by default because eng.traineddata ships with the npm package. For other OCR languages, provide a Tesseract tessdata directory that contains <lang>.traineddata:

const { markdown, meta } = await convert({
    filePath: './scan.pdf',
    langs: ['eng']
});

Configured instances

The default export is zero-config and deterministic-only.

Use createBabel(config) when you want an instance with shared conversion defaults:

import { createBabel } from '@bandf/babel';

const babel = createBabel({
    transliterateHomoglyphs: false,
    ocr: {
        langs: ['eng'],
        tessdataPath: './tessdata'
    }
});

const result = await babel.convert({ filePath: './paper.pdf' });
console.log(result.markdown, result.meta);
await babel.releaseScribe();

Local QA UI

For manual converter QA in this repository:

yarn dev:ui

Open the printed localhost URL, upload a document, and inspect the raw Markdown next to the rendered preview. This server binds to 127.0.0.1 by default and is only intended for local development. The default upload limit is 1.5 GB; override it when needed:

yarn dev:ui --max-upload-size=2gb

Release verification

Run the full release gate before publishing:

yarn verify

That command runs the unit/integration tests, fixture-based core evals, benchmark smoke assertions, and an npm pack dry run. Individual commands are available when iterating:

yarn test
yarn eval
yarn bench
yarn bench:assert

License

AGPL-3.0-or-later. See LICENSE.