hyfnjs
v0.1.0
Published
Neural hyphenation with break scores for 11 languages, in JavaScript
Maintainers
Readme
hyfnjs
A 27,824-parameter neural hyphenator for JavaScript. It predicts word-break scores for 11 languages — English (American), German, French, Spanish, Italian, Dutch, Portuguese, Swedish, Danish, Finnish and Hungarian — and inserts soft hyphens. The model asset is 190 kB (43 kB gzipped); the drop-in script is 20 kB (8 kB gzipped). No dependencies.
For ordinary DOM text in a known language, prefer the browser's hyphens: auto. Reach for this when
you are laying text out yourself (canvas, PDF, custom justification), when you need the break scores
rather than a yes/no, or when the language is not known ahead of time.
Drop it into a page
<script src="https://cdn.jsdelivr.net/npm/hyfnjs/dist/auto.js"></script>That hyphenates the document and keeps hyphenating nodes added later. The model runs in a worker, so
the forward pass stays off the main thread; the script is its own worker payload, so nothing else has
to be hosted. It reads lang attributes for the language and detects it per group when there is none.
Not everything is prose, so script, style, code, pre, kbd, samp, var, textarea,
input, select, option, svg, math, iframe, object, embed and canvas are left alone,
along with [translate="no"], .notranslate and editable regions. data-hyfn="off" opts a subtree
out and data-hyfn="on" opts one back in.
Every option is a data- attribute on the script tag, and every one has a working default:
| Attribute | Default | Meaning |
|---|---|---|
| data-model | ../model.json beside the script | model asset URL |
| data-selector | body | roots to hyphenate |
| data-skip | — | extra CSS selector to leave alone |
| data-lang | read from lang, else detected | force one language |
| data-threshold | 0.725 for English, 0.5 otherwise | only break where p ≥ this |
| data-min-left, data-min-right | 2 | never break this close to either end |
| data-hyphen-char | ­ | what to insert |
| data-observe | true | keep hyphenating nodes added later |
| data-worker | true | run the model in a worker |
The script sets window.hyfn to { ready, config, hyphenate(root?), stop() } and fires hyfn:ready
on document after the first pass, for pages that need to wait for final layout.
Use it as a library
import { load } from "hyfnjs";
const hyphenator = await load("/model.json"); // serve the package's hyfnjs/model.json asset here
hyphenator.hyphenate("typography", { lang: "en" }); // "typog<shy>ra<shy>phy"
hyphenator.breaks("typography", { lang: "en", threshold: 0.9 }); // [{ index, p }, ...]
hyphenator.probabilities("typography"); // one score per code-point gap
hyphenator.hyphenateText("A paragraph to lay out."); // language detected per sentence
hyphenator.detectLang("Silbentrennung verbessert den Blocksatz");load takes a fetchable URL or a parsed model, not a filesystem path. In Node:
import { readFileSync } from "node:fs";
import { fromJSON } from "hyfnjs";
const hyphenator = fromJSON(JSON.parse(readFileSync(new URL(import.meta.resolve("hyfnjs/model.json")), "utf8")));To drive the worker yourself, point a Worker at hyfnjs/worker and wrap it in a RemoteHyphenator,
which must be loaded before inference and rejects failed, timed-out or terminated requests. Node's
worker_threads needs an adapter; the packed-consumer test is a runnable example.
Exports: hyfnjs, hyfnjs/remote, hyfnjs/worker, hyfnjs/auto, hyfnjs/model.json.
What to expect
Offsets are original Unicode code points, not UTF-16 indexes. Text and combining marks are preserved and breaks never split a grapheme. Returned score arrays are frozen. Scores are cached per word, which is what makes a page cheap: a cold word costs ~80 µs, and on real text most words repeat.
A score is not a guarantee of correctness. English is weaker than the other trained languages. Language detection always picks a trained language, so text in an untrained one can get a misleading hint. Norwegian and Czech are evaluated but not trained.
MIT licensed. The weights are trained on words labelled with the hyph-utf8 TeX patterns, which carry their own per-language licences; see NOTICE.
