kabuk-core

The Kabuk engine. Converts PDF, DOCX, PPTX, XLSX, HTML, EPUB, Markdown and plain text to clean Markdown, and reports the token saving.

Most people want kabuk-mcp, which wraps this in an MCP server. Install this one if you are embedding the engine.

The constraint

Core imports no platform API. No fs, no window, no document, no Buffer, no process. Input is bytes plus a hint, output is a string plus metadata.

That is what lets one compiled artifact run in Node, in a browser service worker, and inside JavaScriptCore on macOS. It also means core cannot do a few things for itself, so it takes them by injection instead.

Usage

import { convert, htmlConverter, registerConverter } from "kabuk-core";

// Converters are exported, not self-registering: `sideEffects: false` lets a
// bundler drop a module nobody imports, so registering on import would work in
// Node and silently vanish in a browser build. Register what you need.
registerConverter(htmlConverter);

const result = await convert({
  bytes: new TextEncoder().encode(html),
  filename: "article.html",
});

result.markdown;            // the output
result.delta.savedPercent;  // whole-number percent reduction
result.warnings;            // anything the converter could not do cleanly

Injected capabilities

PDF text extraction. A JS PDF parser costs between 124KB and 490KB gzipped, three to ten times the entire HTML path, and macOS has PDFKit already. So core does the part that matters — reading order, gutter detection, running-header removal — and takes positioned runs from a PdfTextSource you supply. kabuk-mcp ships the Node one.

If you implement PdfTextSource, read PDF_TEXT_SOURCE_CONTRACT, which is exported for exactly that reason. The short version: never merge runs across a horizontal gap. Column detection works by finding a band no run occupies, and a single run spanning two columns has already destroyed the evidence.

OCR. Injected the same way, via OcrProvider. Core never imports one.

Exact token counts. Core ships a segmenting heuristic that lands within roughly five percent on mixed content and labels its output estimated. Register a real BPE for exact:

import { registerTokenizer } from "kabuk-core";
import { encodingForModel } from "js-tiktoken";

const enc = encodingForModel("gpt-4o");
registerTokenizer("gpt", (text) => enc.encode(text).length);

Core depends on no tokenizer package, which keeps a multi-megabyte vocabulary out of a browser bundle.

Stability

From 0.1.0, every type this package exports is semver-frozen. Breaking one needs a major version — including changes that look additive, such as widening a union that appears in a result type, which stops an exhaustive switch compiling on the caller's side.

License

Apache-2.0 · Blu Signal Labs