@x12i/text-entities-transformers
v1.0.0
Published
Optional local English named entity recognition for @x12i/text-entities. The shipped profile is not domain-certified.
Readme
@x12i/text-entities-transformers
Optional local English named entity recognition for @x12i/text-entities.
The shipped profile is xenova-bert-base-ner-candidate. certified is false. Person, organization, and location spans are proposals from Xenova/bert-base-NER at revision 8e892123e8b7c2c0c2bd1dcb598b7d244c4e53aa, quantized to q8. Thresholds are 0.5 and were not tuned on a held-out consumer set. MISC labels are dropped. A model span does not satisfy a topic entity signal and does not override a dictionary mention unless you merge with diagnostics.
describe().extensionAvailable is false. A packaged Chrome extension page was not certified. Domain precision and recall were not measured. See the evaluation report and offline setup.
Install
npm install @x12i/text-entities @x12i/text-entities-transformersRequires Node 22 or newer. Installation does not download weights and does not run a fetch script. prepublishOnly typechecks, tests, and builds. It does not call fetch-model.
Weights stay outside git and outside the npm tarball. Download them with the explicit script in this repository:
npm run fetch-model -w @x12i/text-entities-transformersThat writes models/Xenova/bert-base-NER/ and prints the sha256 of each pinned file. Those hashes are the files list on XENOVA_BERT_BASE_NER_CANDIDATE. A missing file or a mismatch throws ModelUnavailableError (MODEL_UNAVAILABLE) before a session starts. There is no network fallback.
Node
import { createLocalNer, XENOVA_BERT_BASE_NER_CANDIDATE } from "@x12i/text-entities-transformers";
const ner = await createLocalNer({
profile: XENOVA_BERT_BASE_NER_CANDIDATE,
assetRoot: "/absolute/path/models/Xenova/bert-base-NER",
allowNetwork: false,
maxQueuedDocuments: 8,
});
const modeled = await ner.extract(input, { signal });
console.log(ner.describe());
await ner.dispose();allowNetwork must be exactly false. Any other value throws TypeError.
On Node, describe().backend is "cpu". Inference uses the CPU execution provider from onnxruntime-node, one document at a time, inside a worker. WebGPU is not selected. Remote model loading and the Transformers.js cache are turned off, and an http or https fetch from the runtime throws MODEL_UNAVAILABLE.
The input still has to mark English prose. A string with no segments is language und and kind other, so the model skips it with UNSUPPORTED_SEGMENT and status partial. Code, identifiers, and zone key are skipped the same way. A supported segment that cannot be aligned or classified adds SEGMENT_FAILED. If every supported segment fails, status is failed.
Browser
Import the browser entry so the bundle cannot load onnxruntime-node, worker_threads, or node:fs:
import { createLocalNer, XENOVA_BERT_BASE_NER_CANDIDATE } from "@x12i/text-entities-transformers/browser";describe().backend is "wasm". Copy the WASM files next to the model as described in the offline guide. assetRoot must be a file:, chrome-extension:, or relative URL. An http or https root is rejected.
The extension example manifest is examples/extension/manifest.json. Extension pages need script-src 'self' 'wasm-unsafe-eval'. Shipping that manifest is not a certification of the extension page.
Spans
Windows are 384 content tokens with 64 tokens of overlap, clamped to the 512-token model limit minus two special tokens. Windows prefer a sentence boundary and do not cross segments. A span that only touches an artificial window edge is dropped. Exact duplicates collapse. Overlapping spans of different types are both dropped. modelScore is the minimum token score in the span. A span below 0.5 is dropped.
WordPiece alignment walks left to right. A mismatch fails that window. The package does not search for the joined string with indexOf.
Mention ids are assigned after sorting. Provenance records the package, profile, certified: "false", model id, revision, quantization, and backend.
Queue
maxQueuedDocuments defaults to 8 and counts the running document plus waiters. A further call throws QueueFullError (QUEUE_FULL). Nothing is evicted. Aborting the AbortSignal rejects with AbortError and drops a late result. dispose is idempotent, releases the worker, and a later extract throws DisposedError.
Merge
Pass the model result to entities.merge(input, deterministic, modeled) from @x12i/text-entities. Dictionary hits replace overlapping model hits. Validated hits stay beside model hits. A partial model result stays partial after the merge.
License
MIT. Third-party terms are in NOTICE.
