privacy-redactor

v0.1.2

Published

7 days ago

Local-first sensitive information redaction for text, logs, support tickets, and app integrations.

0High
0Medium
0Low

privacy pii redaction secrets logs data-loss-prevention ai-safety flights rail local-first web-component browser-extension

Privacy Redactor / 隐私脱敏器

Local-first sensitive information redaction for text, local documents, logs, support tickets, and app integrations.

在把文本、日志、订单、工单、邮件或文档内容发送给 AI、应用、同事或公开社区之前，先在本地移除敏感信息。

What It Does

Privacy Redactor is a small JavaScript redaction engine that can run in the browser, Node.js, a CLI, a Web Component, or the included browser extension prototype.

It is designed around four rules:

Local-first: redaction runs where the user is, before data crosses a network boundary.
Explainable: every redacted entity reports its type, location, confidence score, and replacement.
Safe metadata by default: raw matched values and surrounding context are opt-in.
Mode-based: apps can choose a narrow mode for fewer false positives or a broad mode for stronger protection.

Modes

| Mode | Purpose | Examples | |---|---|---| | general | Common personal data | email, phone, card number, date of birth, address, SSN, national ID, IPv4/IPv6 address | | travel | Travel and booking data | PNR, booking reference, ticket number, passport number, passenger name, loyalty number, QR payload | | developer | Developer secrets and logs | API keys, passwords, bearer tokens, JWTs, URLs with embedded credentials, OpenAI keys, Google keys, Stripe keys, GitHub tokens, Slack tokens, AWS access key IDs, private key blocks | | finance | Banking and payment operations | IBAN, routing number, bank account, SWIFT/BIC, tax ID, invoice, transaction ID | | healthcare | Patient and insurance workflows | MRN, patient ID, NPI, prescription number, claim number, insurance member ID | | legal | Legal support workflows | case number, docket number, client matter number, court file number, contract number | | hr | HR and employment workflows | employee ID, payroll ID, candidate ID, benefits ID | | education | Student and admissions workflows | student ID, application ID, transcript ID, financial aid ID | | all | Broad default protection | all currently available rule groups |

Default mode: all.

Example

Passenger: PENG/YANG
Booking reference: ABC123
api_key=sk_test_1234567890abcdefghijklmnop
Email: [email protected]
Flight LH455 SFO -> FRA

Passenger: [PASSENGER_NAME]
Booking reference: [BOOKING_REFERENCE]
api_key=[API_KEY]
Email: [EMAIL]
Flight LH455 SFO -> FRA

Install

npm install privacy-redactor

For local development:

npm run verify
npm test
npm run evaluate
npm run smoke:cli
npm run smoke:demo
npm run smoke:extension
npm run smoke:package
npm run check:privacy
npm run check:release
npm run demo

Open http://127.0.0.1:5174/demo/.

Public demo:

https://uilpx.github.io/Privacy-Redactor/demo/

Third-party integration demo:

https://uilpx.github.io/Privacy-Redactor/demo/integration.html

Framework demos:

https://uilpx.github.io/Privacy-Redactor/demo/frameworks.html

Extension test page:

https://uilpx.github.io/Privacy-Redactor/demo/extension-test.html

Library Usage

import { redactText } from "privacy-redactor";

const result = redactText("Email: [email protected] api_key=sk_test_1234567890abcdefghijklmnop", {
  mode: "developer"
});

console.log(result.redactedText);
console.log(result.entities);

TypeScript declarations are included for the main entry point and subpaths such as privacy-redactor/extract, privacy-redactor/component, and privacy-redactor/adapters. Use getAvailableEntityTypes("developer") when an integration needs to build a mode-specific entity picker for includeTypes or excludeTypes. Use validateRedactionOptions(options) before saving user-selected modes, sensitivity presets, entity filters, or custom rules.

Use sensitivity presets when you need a stricter false-positive tradeoff:

redactText(sourceText, { mode: "all", sensitivity: "strict" });
redactText(sourceText, { mode: "all", sensitivity: "balanced" });
redactText(sourceText, { mode: "all", sensitivity: "aggressive" });

Supported Local Files

The CLI and Web Component can read local files and redact extracted text without sending data to a server.

Supported inputs:

Plain text and text-like files: .txt, .md, .csv, .tsv, .json, .xml, .html, .log, .env, .ini, .conf, .sql, .yaml, .yml, .ics, .eml
Rich text: .rtf
Word documents: .docx
PDF: best-effort text extraction from text-based .pdf files

Current limits:

Legacy .doc files are not supported. Save as .docx or export text first.
Scanned PDFs and image-only documents need a local OCR adapter before redaction.
DOCX files can be written to a new redacted .docx file; the original file is never overwritten.
PDF files currently return redacted extracted text; PDF file rewriting is not supported.

Extraction helpers are exported for app integrations:

import { extractTextFromBuffer, redactDocxBuffer, redactStructuredData, redactText } from "privacy-redactor";

const text = await extractTextFromBuffer(bytes, "ticket.docx");
const result = redactText(text, { mode: "all" });

const docx = await redactDocxBuffer(bytes, { mode: "all" });
await saveFileSomewhere(docx.redactedBuffer);

Advanced integrations can pass pdfTextExtractor or ocrTextExtractor adapters for PDF.js, Tesseract.js, native OCR, or another local-only extraction layer. See privacy-redactor/adapters for small adapter factories that accept host-provided PDF.js or Tesseract instances.

Entity reports do not include the original matched value by default:

redactText("Email: [email protected]").entities[0];
// {
//   type: "EMAIL",
//   label: "Email address",
//   start: 7,
//   end: 23,
//   score: 0.98,
//   replacement: "[EMAIL]"
// }

For local debugging only, you can opt in to raw matched values and context:

redactText(sourceText, {
  includeRaw: true,
  includeContext: true
});

Use a specific mode:

redactText(sourceText, { mode: "general" });
redactText(sourceText, { mode: "travel" });
redactText(sourceText, { mode: "developer" });
redactText(sourceText, { mode: "finance" });
redactText(sourceText, { mode: "healthcare" });
redactText(sourceText, { mode: "legal" });
redactText(sourceText, { mode: "hr" });
redactText(sourceText, { mode: "education" });
redactText(sourceText, { mode: "all" });
redactText(sourceText, { mode: ["general", "travel"] });

Customize replacements:

redactText(sourceText, {
  mode: "all",
  numberedPlaceholders: true,
  replacementMap: {
    EMAIL: "[CONTACT]",
    API_KEY: "[SECRET]"
  }
});

Preserve repeat relationships without exposing raw values:

redactText("[email protected] [email protected] [email protected]", {
  includeTypes: ["EMAIL"],
  numberedPlaceholders: true,
  consistentPlaceholders: true
}).redactedText;
// "[EMAIL_1] [EMAIL_2] [EMAIL_1]"

includeTypes and excludeTypes are case-insensitive across the library API, Web Component attributes, and CLI.

Add project-specific rules without forking the package:

redactText("Workspace ID: WS-123456", {
  mode: "general",
  customRules: [
    {
      type: "WORKSPACE_ID",
      label: "Workspace identifier",
      score: 0.87,
      valueGroup: 1,
      pattern: /Workspace ID:\s*(WS-\d{6})/i
    }
  ]
});

Custom rules can use built-in validators, validator functions, or named customValidators. The library automatically runs custom regular expressions globally, so a non-g pattern can still match multiple fields.

Redact structured JSON-like data while keeping object shape:

const result = redactStructuredData({
  user: {
    email: "[email protected]"
  },
  session: {
    token: "short-token"
  }
});

console.log(result.redactedValue);
// { user: { email: "[EMAIL]" }, session: { token: "[SENSITIVE_FIELD]" } }
console.log(result.entities[0].path);
// ["user", "email"]

Structured redaction also redacts values under sensitive keys such as password, token, secret, and apiKey by default. Object keys are used as local detection context, so fields like { phone: "+1 415 555 0123" } can be detected without embedding the label in the value.

Create safe local reports:

import { createRedactionReport, formatRedactionMarkdown, redactText } from "privacy-redactor";

const result = redactText(sourceText, { mode: "all" });
const jsonReport = createRedactionReport(result);
const markdownReport = formatRedactionMarkdown(result);

Reports omit raw matched values and source context by default.

Web Component

With a local module:

<script type="module" src="./src/component.js"></script>

<privacy-redactor mode="all" sensitivity="balanced" numbered-placeholders consistent-placeholders></privacy-redactor>

With a CDN:

<script type="module" src="https://cdn.jsdelivr.net/npm/privacy-redactor/src/component.js"></script>

<privacy-redactor></privacy-redactor>

For production pages, pin a tested version instead of relying on the latest CDN package.

Listen for local redaction results:

document.querySelector("privacy-redactor").addEventListener("redacted", event => {
  console.log(event.detail.redactedText);
  console.log(event.detail.entities);
});

The component does not send data anywhere. It only processes strings in the browser tab. It can open supported local files, extract readable text, and redact the extracted text locally. The input pane shows source name, size, character count, and line count after typing, sample loading, or file import. Detected entities can be reviewed one by one or in bulk, so a user can keep lower-risk values while redacting higher-risk fields. The review list can be filtered by entity type and shows each entity type, replacement, range, confidence label, and confidence explanation. The component can copy redacted text, a safe JSON report, or a Markdown report. Host pages can set mode, sensitivity, include-types, exclude-types, numbered-placeholders, and consistent-placeholders attributes, or assign element.customRules before calling element.redact().

Browser Extension Prototype

The repository root can be loaded as an unpacked Manifest V3 extension for local selected-text redaction.

chrome://extensions -> Developer mode -> Load unpacked -> select this repository root

The extension popup can read the current page selection or focused text field after a user action, redact it locally, write redacted text back to the focused field, and copy redacted text or a safe JSON report. It does not upload text or call remote services.

CLI

node ./bin/privacy-redactor.js --help
node ./bin/privacy-redactor.js --version
node ./bin/privacy-redactor.js --list-types
node ./bin/privacy-redactor.js --mode=developer --list-types
node ./bin/privacy-redactor.js --check ./samples/support-ticket.txt
node ./bin/privacy-redactor.js --config=./samples/privacy-redactor.config.json ./samples/support-ticket.txt
node ./bin/privacy-redactor.js ./samples/flight-booking.txt
node ./bin/privacy-redactor.js ./document.docx
node ./bin/privacy-redactor.js ./ticket.pdf
node ./bin/privacy-redactor.js ./samples/industry-records.txt
node ./bin/privacy-redactor.js --mode=developer ./logs.txt
node ./bin/privacy-redactor.js --mode=all --sensitivity=strict ./support-ticket.txt
node ./bin/privacy-redactor.js --include-types=EMAIL,API_KEY ./support-ticket.txt
node ./bin/privacy-redactor.js --exclude-types=IP_ADDRESS ./support-ticket.txt
node ./bin/privacy-redactor.js --include-types=EMAIL --numbered-placeholders --consistent-placeholders ./ticket.txt
node ./bin/privacy-redactor.js ./document.docx --output=./document.redacted.docx
node ./bin/privacy-redactor.js --structured ./payload.json
node ./bin/privacy-redactor.js --jsonl ./events.jsonl
node ./bin/privacy-redactor.js --format=markdown ./support-ticket.txt
node ./bin/privacy-redactor.js --format=json ./support-ticket.txt
node ./bin/privacy-redactor.js --json --mode=all ./support-ticket.txt
node ./bin/privacy-redactor.js --json --include-raw ./support-ticket.txt
node ./bin/privacy-redactor.js -- -dash-prefixed-file.txt
cat ./samples/flight-booking.txt | node ./bin/privacy-redactor.js

--format=json and --format=markdown produce safe reports without raw matched values by default. --json is kept for compatibility and returns the raw result object. --help prints available modes and options. --version prints the installed package version. --list-types prints entity types available for the selected mode. --check prints a safe detection summary without raw values or redacted source text, then exits with status 1 if sensitive fields are found and 0 if none are found. --config=path/to/privacy-redactor.json reads local JSON defaults for mode, sensitivity, format, includeTypes, excludeTypes, numberedPlaceholders, consistentPlaceholders, and local output flags. Explicit CLI option values override matching config values. See samples/privacy-redactor.config.json for a bundled starter config. --include-types and --exclude-types accept comma-separated entity types and work across text, DOCX output, structured JSON, and JSONL modes. Type names are case-insensitive; unknown type names fail fast with a --list-types hint. --numbered-placeholders and --consistent-placeholders expose the same repeat-preserving placeholders as the library API. --structured parses input as JSON and prints the redacted JSON value by default; combine it with --format=json for a structured report with entity paths. --jsonl parses newline-delimited JSON and prints one redacted JSON value per line; combine it with --format=json for per-record line numbers and entity paths. Use -- before an input file path that starts with a dash. Unknown options, empty option values, and multiple input files fail fast with a usage hint.

Rule Evaluation

Synthetic regression fixtures can be checked independently from the full test suite:

npm run evaluate

Use this before publishing rule changes or reviewing false-positive / false-negative contributions. Use npm run verify before a release to run tests, fixture evaluation, CLI smoke tests, demo smoke tests, extension smoke tests, package install smoke tests, documentation checks, privacy boundary checks, release consistency checks, and package dry-run together.

Project Goals / 项目目标

Build a general-purpose local redaction engine that other apps can embed.
Make pre-send privacy review normal for AI workflows.
Provide useful built-in modes without locking the project to one industry.
Keep the core dependency-free and easy to audit.
Preserve non-sensitive utility where possible, such as flight numbers, route facts, log structure, timestamps, and error messages.

Implementation Approach / 实现方法

Privacy Redactor currently uses deterministic local detectors:

Lightweight local text extraction for text-like files, .rtf, .docx, and text-based .pdf.
Regular expressions for context-bound identifiers.
Luhn validation for payment cards.
Plausibility validators for dates, SSNs, China resident identity card numbers, identifiers, and generic secrets.
Rule groups mapped to modes.
Industry-specific context-bound modes for finance, healthcare, legal, HR, and education.
Sensitivity presets for stricter or broader detection thresholds.
Overlap resolution that keeps the highest-confidence entity.
Structured output for UI review, logs, and app integration.

Future adapters can add stronger PDF parsing, local OCR, locale packs, and optional ML/NLP detectors. Remote processing should remain opt-in and outside the core package.

Explicit Non-Goals For v0.1

It is not a legal compliance product.
It does not guarantee complete sensitive information removal.
It only redacts strings or locally extracted text.
It does not modify binary PDFs, office documents, or images in place.
It does not perform cloud OCR.
It does not use remote AI services.

Documentation

Repository Name

Recommended GitHub repository:

privacy-redactor

Recommended npm package: