Rampart

Rampart is a local-first system for removing personally identifiable information from user-typed text before it leaves the browser. It combines a 14.7 MB ONNX token-classification model with a deterministic recognizer layer; together they form a defense-in-depth pipeline released as a complete, reproducible artifact.

This repository ships the runtime as @nationaldesignstudio/rampart. Model weights load from Hugging Face by default; a copy also lives in model/ for local serving and training.

"My name is Alex Rivera and my SSN is 472-81-0094."
→ "My name is [GIVEN_NAME_1] [SURNAME_1] and my SSN is [SSN_1]."

The model provider sees placeholders. The user sees restored values on the client. The session table never leaves the device.

Rampart is harm reduction, not perfect protection. Personal data the client fails to redact is the upper bound on what can leak through a model provider, a logging pipeline, or a future infrastructure compromise. No detector at this size catches everything; we document failure modes openly and ship regression tests so future training runs surface drops immediately.

See WHITEPAPER.md for the full technical writeup — methodology, candidate sweep, calibration, schema reconciliation, and reproducibility.

Supported scope

This release supports seven Latin-script languages: English, Spanish, French, German, Italian, Portuguese, and Dutch. Every headline number below is measured on these languages.

Text and names in non-Latin scripts (e.g. Chinese, Japanese, Korean, Arabic, Cyrillic, Devanagari) are out of scope in this release: recall drops sharply and the system should not be relied on for them. See Limitations.

Headline results

On a 30,000-row held-out test set spanning all seven supported languages from the OpenPII 1.5M dataset, the full system (model + deterministic layer + policy) achieves:

| Metric | Value | | --- | ---: | | Private-term recall (7 languages) | 98.42% (Wilson 95% CI [98.35, 98.49]) | | Public-term retention | 91.7% term-presence (>99% policy-aware*) | | Latency p50 (Node ONNX) | 6.6 ms† | | Shipped artifact size | 14.7 MB Q4 ONNX (≈15.0 MB with tokenizer) |

* OpenPII marks street-line components as public; Rampart redacts the precise street line (BUILDING_NUMBER + STREET_NAME) and the secondary-address line (SECONDARY_ADDRESS) while keeping city, state, and ZIP. See "Schema reconciliation" in the whitepaper.

† Latency is hardware-dependent; the committed eval/bench/runs/sample-900 proof run records ≈14 ms p50 on CI hardware. Over the held-out slice the browser pipeline runs at 3.9 ms p50 on WebGPU (Apple Metal) and 12.6 ms on WASM via bun run bench:webgpu.

These numbers come from the shipped Q4 pipeline scored end-to-end by the committed eval/bench harness on a pinned held-out slice. The harness was corrected since the previous revision: city/state/ZIP are now scored as kept (matching the runtime policy) rather than counted as leaks, which is why public retention rose to ~90% while the headline recall is reported against the larger, harder seven-language slice.

Per supported language, on the same 30,000-row test set:

| Language | Private recall | Public retention | | --- | ---: | ---: | | English (en) | 98.85% | 90.5% | | Spanish (es) | 98.84% | 91.6% | | French (fr) | 98.41% | 92.8% | | German (de) | 97.94% | 91.7% | | Italian (it) | 97.83% | 94.1% | | Portuguese (pt) | 97.73% | 92.5% | | Dutch (nl) | 97.21% | 91.9% |

On the English+Spanish slice the full system scores 98.85% recall — the slice used for the model-selection sweep in the whitepaper.

Design goals

1. Local-first privacy. Remove personal information before it reaches application infrastructure. Data the server never receives cannot be leaked downstream.
2. Browser-deployable. Under 15 MB on the wire — small enough for a low-end phone over a slow connection.
3. Recall-biased. A miss leaks data; over-redaction is the lesser failure mode.
4. Domain-aware retention. The keep-set is policy-driven so assistants retain coarse geography — city, state, ZIP — while the precise street line is redacted, all without retraining.

Architecture

Two cooperating layers run in parallel and merge their outputs. Both run entirely in the browser.

Deterministic recognizer layer

Regular expressions paired with checksum and structural validators. It owns five classes end-to-end:

• Credit cards — Luhn-checksummed over the digit projection, so every separator form collapses to one rule and a 16-digit number that fails Luhn is not redacted as a card.
• SSNs — structural rules reject reserved areas (000, 666, 9xx) and ZIP+4-style false positives.
• Email, URLs, and IP addresses (IPv4, IPv6, and MAC) — pattern-backed, where the structure lives in the punctuation; near-100% recall, far above the model alone (model-only URL recall is ~5%).

This layer is synchronous and runs before the model loads; its spans are masked to sentinels so the model never re-derives them. Names, phone numbers, account and routing numbers, government identifiers, passports, licenses, and street-address components carry no checksum, so they are left to the model rather than guessed at with a regex.

Token-classification model

A MiniLM-L6-H384 encoder fine-tuned on a 35-label BIO head (17 entity types) covers contextual PII the regex layer can't checksum — split names (GIVEN_NAME, SURNAME), phone numbers, account/routing/tax numbers, government IDs, passports, licenses, and free-form address components — across seven Latin-script languages (en, es, fr, de, it, pt, nl). Vocabulary is trimmed to 19,730 WordPieces; single-character pieces are retained for rare-name fallback.

Span repair (adjacent merge, bridge-and-merge, capitalized-particle rescue) lifts span-F1 to 0.53 strict (IoU=1.0) and 0.66 relaxed (IoU≥0.5) on the headline test set — well above the fragmented spans HuggingFace's default aggregation produces for subword-split names.

Policy and session table

Default-deny policy: every detected label is redacted unless explicitly kept. The default keep-set is {CITY, STATE, ZIP_CODE} — coarse geography an assistant can reason about — while the precise street line (BUILDING_NUMBER + STREET_NAME) and the secondary-address line (SECONDARY_ADDRESS) are always redacted.

Session table: maps each raw value to a stable placeholder (Maria Garcia → [GIVEN_NAME_1] [SURNAME_1]). Placeholders are restored locally in assistant responses. The table is never transmitted.

The npm package exposes a single entry point — createGuard() returns a ChatGuard that runs the full pipeline (detect → policy → placeholders) and keeps per-conversation state for reveal().

Install

npm install @nationaldesignstudio/rampart @huggingface/transformers

@huggingface/transformers is a peer dependency — your app bundles and serves it.

Usage

Create one ChatGuard per conversation. createGuard() loads the shipped q4 classifier (nationaldesignstudio/rampart) from Hugging Face by default, caches it on-device, and pairs it with the deterministic layer.

import { createGuard } from "@nationaldesignstudio/rampart";

const guard = await createGuard();

// Scrub the user message before it reaches your LLM or server.
const safe = await guard.protect(userMessage);

// Send safe.text — placeholders, not raw PII — to the model.
const reply = await llm(safe.text);

// Restore real values before showing the reply to the user.
guard.reveal(reply);

Streaming replies: stream.pipeThrough(guard.revealTransform()).

Scrub model output before logging: await guard.protectReply(reply).

Options

| Option | Default | Purpose | | --- | --- | --- | | model | nationaldesignstudio/rampart | Hugging Face model id or local ONNX directory | | device | "wasm" | "wasm" / "webgpu" in browsers; "cpu" in Node | | worker | — | Worker script URL — run NER off the main thread | | heuristicsOnly | false | Skip the classifier; structured PII only | | keepLabels | city, state, ZIP | Widen or narrow the default-deny keep-set | | aliases | {} | Display names for tokens, e.g. { GIVEN_NAME: "NAME" } | | ner | — | Inject a custom detector; skips model | | minScore | 0.4 | Drop model spans below this confidence | | noPrefilter | false | Feed raw text to the model (no-prefilter ablation); heuristics still run |

// Local weights (e.g. self-hosted or repo `model/` dir)
const guard = await createGuard({ model: "./model", device: "cpu" });

// Heuristics only — no model load
const guard = await createGuard({ heuristicsOnly: true });

// Keep inference off the UI thread (browser)
const guard = await createGuard({
  worker: new URL("./pii-worker.ts", import.meta.url),
});

Custom models must be token-classification ONNX exports (q4) with a label schema compatible with Rampart. Dtype is fixed to q4.

Set HF_TOKEN when pulling from a private Hugging Face repo.

CLI

bun run redact   # interactive terminal redactor (Node, device: cpu)

Limitations

The most consequential documented gaps:

• Non-Latin scripts are out of scope. This release supports the seven Latin-script languages above only. On the fairness suite, Latin-script names — including diacritics — recall ~99.8%, but names in non-Latin scripts recall ~14% in aggregate (Russian 2%, Arabic 5%, Hindi 6%, Han Chinese 9%, Korean 15%, Japanese 46%). There is no checksum for names, so this gap surfaces at the system level. Do not deploy this release for populations who routinely type non-Latin-script names without compensating controls. Tracked by a stratified regression test in the eval suite; closing it is the top priority for the next training cycle.
• Adversarial robustness. 86.4% on a 20-case hostile-input suite. Combined attacks can still bypass both layers. The threat model is good-faith user entry, not a motivated adversary smuggling PII past their own filter.
• Indirect identifiers. Rare condition + ZIP-style inferential leaks are out of scope.
• Non-text inputs. Images, audio, and structured form fields are not supported.

See MODEL_CARD.md for per-class statistics and failure modes.

Evaluation

Everything is evaluated in TypeScript against the shipped ChatGuard pipeline — the same code consumers run is the code under test:

• Unit tests — deterministic detectors, redaction policy, streaming rehydration, span repair.
• Public API end-to-end suite — chat-style cases across structured identifiers, names from multiple traditions, addresses, government IDs, keep-set behavior, and no-PII controls.
• Native benchmark (eval/bench) — runs the real @nationaldesignstudio/rampart pipeline over a frozen OpenPII held-out slice and scores recall/retention (Wilson CI), span-F1, and latency. The headline numbers above are regenerated by this harness.

bun test                                   # unit + public API suites
bun run eval:public                        # end-to-end chat cases

bun run bench:fetch --n 30000              # materialise the held-out slice
bun run bench                              # score the shipped pipeline

Documentation

| Document | Contents | | --- | --- | | WHITEPAPER.md | Full technical writeup | | MODEL_CARD.md | Model summary, training data, eval results, limitations | | RELEASE.md | Verify and publish checklist |

Distribution

| Channel | Artifact | | --- | --- | | npm | @nationaldesignstudio/rampart — TypeScript runtime API | | GitHub | nationaldesignstudio/rampart — source, tests, eval harness, model weights |

License

Released under CC BY 4.0 (Creative Commons Attribution 4.0 International).

Training data: OpenPII 1.5M (CC BY 4.0).