@xberg-io/xberg-wasm
v1.4.0
Published
High-performance document intelligence library
Downloads
1,748
Readme
WebAssembly
Extract text, tables, images, and metadata from 107 file formats including PDF, Office documents, and images. WebAssembly bindings for browsers, Deno, and Cloudflare Workers with a single-threaded synchronous core and an async JavaScript surface.
What This Package Provides
- Document intelligence core — extract text, tables, images, metadata, entities, keywords, and code intelligence through the shared Rust engine.
- Format coverage — PDF, Office, images, HTML/XML, email, archives, notebooks, citations, scientific formats, and plain text.
- OCR support — Tesseract WASM when OCR is enabled.
- Pure-Rust ML inference — RT-DETR layout detection and document-orientation run through the pure-Rust
tractengine (detectLayout/detectOrientation, with the.onnxweights streamed in). This build links no ONNX Runtime, so PaddleOCR, embeddings, reranking, and native transcription are not included. - Same engine as every binding — Rust, Python, Node.js, Go, Java, PHP, Ruby, .NET, Elixir, WASM, Kotlin Android, Swift, Dart, Zig, and C FFI share the same Rust implementation.
- WASM package — browser and edge-compatible extraction where native libraries are unavailable.
Installation
Package Installation
pnpm add @xberg-io/xberg-wasmSystem Requirements
- Modern browser with WebAssembly support, or Deno 1.0+, or Cloudflare Workers
- Optional: Tesseract WASM for OCR functionality
Quick Start
Basic Extraction
Extract text, metadata, and structure from any supported document format:
import init, { WasmExtractInputKind, extract } from "@xberg-io/xberg-wasm";
async function main() {
await init();
const buffer = await fetch("document.pdf").then((r) => r.arrayBuffer());
const bytes = new Uint8Array(buffer);
const output = await extract({
kind: "bytes",
bytes,
mimeType: "application/pdf",
filename: "document.pdf",
}, undefined);
console.log("Extracted content:");
console.log(output.results[0].content);
console.log("MIME type:", output.results[0].mimeType);
console.log("Metadata:", output.results[0].metadata);
}
main().catch(console.error);Common Use Cases
Extract with Custom Configuration
Most use cases benefit from configuration to control extraction behavior:
With OCR (for scanned documents):
import init, { WasmExtractInputKind, extract } from "@xberg-io/xberg-wasm";
async function extractWithOcr() {
await init();
const buffer = await fetch("scanned-page.png").then((response) => response.arrayBuffer());
const bytes = new Uint8Array(buffer);
// OCR is turned on per extraction through the `ocr` config block. There is no
// separate global "enable OCR" call — the backend is selected by name here.
const output = await extract(
{
kind: "bytes",
bytes,
mimeType: "image/png",
filename: "scanned-page.png",
},
{
ocr: {
enabled: true,
backend: "tesseract",
language: ["eng"],
},
},
);
console.log("Extracted text:");
console.log(output.results[0].content);
}
extractWithOcr().catch(console.error);Table Extraction
See Configuration Guide for table extraction options.
Processing Multiple Files
import init, { WasmExtractInput, extractBatch } from "@xberg-io/xberg-wasm";
interface DocumentJob {
name: string;
bytes: Uint8Array;
mimeType: string;
}
async function _processBatch(documents: DocumentJob[], concurrency: number = 3) {
await init();
const results: Record<string, string> = {};
for (let index = 0; index < documents.length; index += concurrency) {
const batch = documents.slice(index, index + concurrency);
const output = await extractBatch(
batch.map((doc) => WasmExtractInput.fromBytes(doc.bytes, doc.mimeType, doc.name)),
undefined,
);
output.results.forEach((result, resultIndex) => {
results[batch[resultIndex].name] = result.content ?? "";
});
}
return results;
}Async Processing
For non-blocking document processing:
import init, { extract } from "@xberg-io/xberg-wasm";
async function extractDocuments(files: Uint8Array[], mimeTypes: string[]) {
await init();
const results = await Promise.all(
files.map((bytes, index) => extract({ kind: "bytes", bytes, mimeType: mimeTypes[index] }, undefined)),
);
return results.map((r) => ({
content: r.results[0].content,
metadata: r.results[0].metadata,
}));
}
const fileBytes = [new Uint8Array([1, 2, 3])];
const mimes = ["application/pdf"];
extractDocuments(fileBytes, mimes)
.then((results) => console.log(results))
.catch(console.error);Next Steps
- Installation Guide - Platform-specific setup
- API Documentation - Complete API reference
- Examples & Guides - Full code examples and usage guides
- Configuration Guide - Advanced configuration options
Features
Supported File Formats (107 formats · 141 file extensions · 56 MIME aliases)
107 formats across 140 unique file extensions, with 56 compatibility MIME aliases, intelligent format detection, and comprehensive metadata extraction.
Office Documents
| Category | Formats | Capabilities |
|----------|---------|--------------|
| Word Processing | .docx, .docm, .doc, .dotx, .dotm, .dot, .odt, .pages, .wpd, .wp, .wp5, .wp6 | Full text, tables, images, metadata, styles |
| Spreadsheets | .xlsx, .xlsm, .xlsb, .xls, .xla, .xlam, .xltm, .xltx, .xlt, .ods, .numbers | Sheet data, formulas, cell metadata, charts |
| Presentations | .pptx, .pptm, .ppt, .pps, .ppsx, .potx, .potm, .pot, .odp, .key | Slides, speaker notes, images, metadata |
| PDF | .pdf | Text, tables, images, metadata, OCR support |
| eBooks | .epub, .fb2 | Chapters, metadata, embedded resources |
| Database | .dbf, .sqlite, .sqlite3, .db, .gpkg, .gpkx | Bounded table extraction, schema metadata, GeoPackage detection |
| Hangul | .hwp, .hwpx | Korean document format, text extraction |
Images (OCR-Enabled)
| Category | Formats | Features |
|----------|---------|----------|
| Raster | .png, .jpg, .jpeg, .gif, .webp, .bmp, .tiff, .tif | OCR, table detection, EXIF metadata, dimensions, color space |
| Advanced | .jp2, .jpg2, .j2c, .j2k, .jpc, .jbig2, .jb2, .pnm, .pbm, .pgm, .ppm | OCR via hayro-jpeg2000 (pure Rust decoder), JBIG2 support, table detection, format-specific metadata |
| Vector | .svg | DOM parsing, embedded text, graphics metadata |
Web & Data
| Category | Formats | Features |
|----------|---------|----------|
| Markup | .html, .htm, .xhtml, .xht, .xml, .kml, .svg | DOM parsing, metadata (Open Graph, Twitter Card), link extraction |
| Structured Data | .json, .geojson, .jsonl, .ndjson, .yaml, .yml, .toml, .csv, .tsv | Schema detection, nested structures, validation |
| Text & Markdown | .txt, .adoc, .asciidoc, .vtt, .md, .markdown, .commonmark, .qmd, .rmd, .djot, .dj, .mdx, .doctags, .rst, .org, .rtf | AsciiDoc, CommonMark, MyST Markdown, Quarto, R Markdown, Djot, MDX, DocTags, reStructuredText, Org Mode |
Email & Archives
| Category | Formats | Features |
|----------|---------|----------|
| Email | .eml, .msg, .pst | Headers, body (HTML/plain), attachments, threading |
| Archives | .zip, .tar, .tgz, .gz, .7z | Recursive extraction of nested archives, file listing, metadata, zip-bomb protection |
Academic & Scientific
| Category | Formats | Features |
|----------|---------|----------|
| Citations | .bib, .ris, .nbib, .enw | Structured parsing: RIS, PubMed/MEDLINE, EndNote XML, BibTeX/BibLaTeX, CSL JSON by MIME type |
| Scientific | .tex, .latex, .typ, .typst, .jats, .nxml | LaTeX, Typst, PubMed JATS |
| Text notebooks | .ipynb, .md, .py, .R, .jl | Jupyter, MyST-NB, Jupytext percent/light, saved outputs, cell visibility tags |
| Publishing | .fb2, .docbook, .dbk, .docbook4, .docbook5, .opml | FictionBook, DocBook XML, OPML outlines |
Code Intelligence (371 Languages)
| Feature | Description | |---------|-------------| | Structure Extraction | Functions, classes, methods, structs, interfaces, enums | | Import/Export Analysis | Module dependencies, re-exports, wildcard imports | | Symbol Extraction | Variables, constants, type aliases, properties | | Docstring Parsing | Google, NumPy, Sphinx, JSDoc, RustDoc, and 10+ formats | | Diagnostics | Parse errors with line/column positions | | Syntax-Aware Chunking | Split code by semantic boundaries, not arbitrary byte offsets |
Powered by tree-sitter-language-pack — documentation.
Key Capabilities
- Text Extraction - Extract all text content with position and formatting information
- Metadata Extraction - Retrieve document properties, creation date, author, etc.
- Table Extraction - Parse tables with structure and cell content preservation
- Image Extraction - Extract embedded images and render page previews
- OCR Support - Integrate multiple OCR backends for scanned documents
- Async/Await - Non-blocking document processing with concurrent operations
- Plugin System - Extensible post-processing for custom text transformation
- Batch Processing - Efficiently process multiple documents in parallel
- Memory Efficient - Stream large files without loading entirely into memory
- Language Detection - Detect and support multiple languages in documents
- Configuration - Fine-grained control over extraction behavior
- Six Output Formats - Plain text, Markdown, Djot, HTML, JSON tree structure, or Docling DocTags
OCR Support
Xberg supports multiple OCR backends for extracting text from scanned documents and images:
Tesseract-Wasm
Sceptre (Opt-In Worker Api)
OCR Configuration Example
import init, { WasmExtractInputKind, extract } from "@xberg-io/xberg-wasm";
async function extractWithOcr() {
await init();
const buffer = await fetch("scanned-page.png").then((response) => response.arrayBuffer());
const bytes = new Uint8Array(buffer);
// OCR is turned on per extraction through the `ocr` config block. There is no
// separate global "enable OCR" call — the backend is selected by name here.
const output = await extract(
{
kind: "bytes",
bytes,
mimeType: "image/png",
filename: "scanned-page.png",
},
{
ocr: {
enabled: true,
backend: "tesseract",
language: ["eng"],
},
},
);
console.log("Extracted text:");
console.log(output.results[0].content);
}
extractWithOcr().catch(console.error);Async Support
This binding provides full async/await support for non-blocking document processing:
import init, { extract } from "@xberg-io/xberg-wasm";
async function extractDocuments(files: Uint8Array[], mimeTypes: string[]) {
await init();
const results = await Promise.all(
files.map((bytes, index) => extract({ kind: "bytes", bytes, mimeType: mimeTypes[index] }, undefined)),
);
return results.map((r) => ({
content: r.results[0].content,
metadata: r.results[0].metadata,
}));
}
const fileBytes = [new Uint8Array([1, 2, 3])];
const mimes = ["application/pdf"];
extractDocuments(fileBytes, mimes)
.then((results) => console.log(results))
.catch(console.error);Plugin System
Xberg supports extensible post-processing plugins for custom text transformation and filtering.
For detailed plugin documentation, visit Plugin System Guide.
Batch Processing
Process multiple documents efficiently:
import init, { WasmExtractInput, extractBatch } from "@xberg-io/xberg-wasm";
interface DocumentJob {
name: string;
bytes: Uint8Array;
mimeType: string;
}
async function _processBatch(documents: DocumentJob[], concurrency: number = 3) {
await init();
const results: Record<string, string> = {};
for (let index = 0; index < documents.length; index += concurrency) {
const batch = documents.slice(index, index + concurrency);
const output = await extractBatch(
batch.map((doc) => WasmExtractInput.fromBytes(doc.bytes, doc.mimeType, doc.name)),
undefined,
);
output.results.forEach((result, resultIndex) => {
results[batch[resultIndex].name] = result.content ?? "";
});
}
return results;
}Configuration
For advanced configuration options including language detection, table extraction, OCR settings, and more:
Documentation
Contributing
Contributions are welcome! See Contributing Guide.
Part of Xberg.io
- Xberg — the open-source content-intelligence engine: text, tables, and metadata from 107 formats (141 file extensions), with OCR, transcription, and code intelligence. MIT.
- Xberg Pro — a complete self-hosted content-intelligence backend in a single container. Commercial.
- Xberg Enterprise — the distributed, governed content-intelligence platform, scaled on Kubernetes with team governance and support. Commercial.
- crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
- html-to-markdown — fast, lossless HTML→Markdown engine.
- liter-llm — universal LLM API client with native bindings for 14 languages and 165 providers.
- tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
- alef — the polyglot binding generator that produces this README and all per-language bindings.
- Discord — community, roadmap, announcements.
License
MIT License — see LICENSE for details.
Support
- Discord Community: Join our Discord
- GitHub Issues: Report bugs
- Discussions: Ask questions
