npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@webywz/context-diet

v0.13.0

Published

Slim LLM context before you send it — drop noise, keep the signal, estimate tokens.

Readme

context-diet

npm node license

Slim LLM context before you send it.

把要喂给模型的上下文先瘦身。

Drop noise, keep signal, estimate tokens — locally, offline, in milliseconds.

丢掉噪声,留下信号,估算 token —— 本地、离线、毫秒级。


English

Demo

npx @webywz/context-diet --demo --preset logs --max-tokens 400

Bundled examples/noisy.txt goes in; slimmed body on stdout, human report on stderr. No clone required — examples/ ships in the npm package.

context-diet 0.5.0
source:   examples/noisy.txt
chars:    1,666 → 834  (−49.9%)
tokens:   ~667 → ~312  (−53.2%)
cut:
  boilerplate lines     9
  duplicate paragraphs  2
  omitted heavy blobs   1
  …

Features

| | | | --- | --- | | Offline heuristics | Strip ANSI/HTML chrome, drop boilerplate, dedupe, omit base64 blobs | | Presets | logs, email, chat, html, code, json — sensible keep/strip defaults | | Budget | --max-tokens drops low-value blocks first (shared across files/dirs); --keep pins headings/code/lists | | Secret redaction | --redact scrubs PEM / JWT / sk- / ghp_ / Slack / AWS / Bearer / Basic → [redacted] | | CI token gate | --fail-over-tokens <n> exits 1 when combined output tokens still > n (shared multi-file budget; no silent truncate); --sarif <path> for Code Scanning; --tap for TAP 13; JSON budget field for Actions | | Pipe-friendly | Slimmed body → stdout; report → stderr | | Bundled examples | --demo / examples/noisy.txt in the published tarball | | Optional deepen | --deepen LLM second pass only when you opt in with a key | | HTML report | --html <path> writes a self-contained before/after report (offline, no CDN) | | Multi-file / dirs | Pass several files or directories; they share one --max-tokens / --fail-over-tokens budget (budget.shared, sources[]). Directories stay top-level unless -R / --recursive | | Step summary | --github-summary appends a short diet report to $GITHUB_STEP_SUMMARY |

Install

npm i -g @webywz/context-diet

Node 18+. No tokenizer files, no GPU, no account.

npx @webywz/context-diet --help

Quick start

context-diet input.txt
npx @webywz/context-diet --demo --preset logs --max-tokens 400
cat noisy.log | context-diet --preset logs --max-tokens 1500 > slim.txt
context-diet notes.md -o slim.md --keep headings,code,lists
context-diet input.txt --dry-run --diff
context-diet --demo --html diet.html -q
context-diet input.txt --json | jq '{tokens: .output.tokens, cuts}'
context-diet dump.txt --redact --fail-over-tokens 2000
context-diet a.md b.md notes/ --max-tokens 2000 --fail-over-tokens 2500 --json
context-diet --demo --fail-over-tokens 1 --sarif over.sarif
context-diet examples/noisy.txt --keep headings|code|lists

Presets

| Preset | Keeps by default | Extra strippers | | --- | --- | --- | | logs | code | Prefer ERROR / WARN / FATAL when truncating | | email | headings, lists | Quoted replies (> / On … wrote: / Original Message) | | chat | code, lists | Reactions, join/leave, lone timestamps | | html | headings, code, lists | <script> / <style> / comments / tags → prose | | code | code | Long comments / blank runs; prefer signatures when truncating | | json | code | JSONC comments out; drop null / ""; stable pretty-compact |

Explicit --keep always wins over the preset default.

What it does

All offline unless you pass --deepen.

| Step | Behavior | | --- | --- | | Redact secrets | Optional --redact before other steps | | Strip ANSI | Terminal color / OSC sequences | | Strip HTML | Optional / --preset html | | Omit heavy blobs | Long base64 / data-URIs → [omitted blob: N chars] | | Collapse whitespace | Never inside code fences | | Drop boilerplate | Repeated log lines, signatures, DEBUG chrome | | Drop quoted / chat chrome | Email quotes; reactions / join-leave | | Dedupe paragraphs | Exact and near-identical (Jaccard ≥ 0.84) | | Truncate | --max-tokens + --keep / presets | | Fail over tokens | --fail-over-tokens hard CI gate on combined output tokens after diet (shared across inputs) | | TAP | --tap TAP version 13 (exit 1 when over --fail-over-tokens) | | JSON budget | budget.failOverTokens / budget.outputTokens / budget.ok / budget.shared / sources[] in --json |

Token estimates are cl100k-ish (~4 chars/word, 1 CJK ≈ 1 token). Good enough to budget a prompt — not a billed tokenizer.

CLI

context-diet [file|dir ...] [options]

| Flag | Meaning | | --- | --- | | [file\|dir ...] | One or more paths; dirs expand to top-level files unless --recursive; omit or - for stdin. Multi-input shares one token budget | | -o, --output <file> | Write slimmed text here instead of stdout | | --html <path> | Write self-contained HTML report (stats + capped previews) | | --max-tokens <n> | Cap estimated output tokens (truncates; shared across files/dirs) | | --fail-over-tokens <n> | Exit 1 if combined output tokens still > n (CI gate; shared; distinct from truncate) | | --sarif <path> | Write SARIF 2.1.0 when over --fail-over-tokens (GitHub Code Scanning) | | --tap | TAP version 13 on stdout (CI; exit 1 if over --fail-over-tokens) | | --redact | Scrub common secret shapes → [redacted] (before diet heuristics) | | --keep <cats> | Prefer headings, code, lists when truncating (comma or \|) | | --preset <name> | logs | email | chat | html | code | json | | --exclude <glob> | Skip files matching glob pattern (repeatable; dirs only) | | --include <glob> | Only keep paths matching at least one glob (repeatable; dirs only) | | --max-files <n> | Fail (exit 3) when a dir scan would process > n files after filters | | -R, --recursive | Also include files in nested directories. Default stays top-level only. Does not follow directory symlinks | | --demo | Diet bundled examples/noisy.txt | | --json | JSON report on stdout | | --dry-run | Report only; no body / no -o write | | --diff | Show removed lines on stderr (capped) | | --deepen | Optional LLM rewrite | | --provider openai\|anthropic | Force provider for --deepen | | --github-summary [path] | Append diet Markdown to $GITHUB_STEP_SUMMARY or path | | -q, --quiet | No human report on stderr | | -h, --help / -V, --version | Help / version |

Default I/O stays pipe-friendly:

pbpaste | context-diet --preset chat --max-tokens 800 | pbcopy

Directory scanning

Directories expand to top-level files unless you pass -R or --recursive. Recursive mode also selects files in nested directories. It does not follow directory symlinks, so a link cannot loop or walk outside the tree. Names starting with . are skipped at every level.

When scanning directories, context-diet automatically skips:

  • node_modules/ and .git/ (always)
  • Patterns from .gitignore (when found in project root or nearest git root)
  • Custom patterns via --exclude <glob> (repeatable)
  • Hidden names (prefix .)

You can also filter with include patterns (only files matching at least one --include glob are kept). Globs match the path relative to the directory you passed, and * does not cross /. --exclude '*.tmp' also matches that pattern against any single path segment, so it skips nested *.tmp files. --include '*.ts' matches only a top-level *.ts path; with --recursive, use **/*.ts to keep nested TypeScript files.

context-diet src/ --include '*.ts' --include '*.js'
context-diet src/ --exclude '*.tmp' --exclude 'test-*'
context-diet . --max-tokens 2000
context-diet src/ --recursive --include '**/*.ts' --max-files 100

For safety, use --max-files <n> to prevent processing too many files. The limit is applied after include, exclude, gitignore, and (when set) recursive expansion:

context-diet docs/ --max-files 100              # exits 3 if > 100 top-level files
context-diet docs/ --recursive --max-files 100  # exits 3 if > 100 files after the nested walk

Optional LLM pass

--deepen is never required. With a key:

export OPENAI_API_KEY=sk-...          # or ANTHROPIC_API_KEY
context-diet chapter.md --deepen --max-tokens 1200

Defaults: gpt-4o-mini / claude-3-5-haiku-20241022 (OPENAI_MODEL / ANTHROPIC_MODEL). Local result wins if the model returns longer text. Without a key, --deepen exits 2 and does not phone home.

CI (GitHub Actions)

Starter: examples/github-action.yml (includes --github-summary).

- uses: actions/setup-node@v4
  with:
    node-version: 22
- name: Slim context
  run: |
    npx --yes @webywz/context-diet --demo --preset logs --max-tokens 400 \
      --redact --fail-over-tokens 500 > slim.txt

Library

import { diet, dietAsync, estimateTokens } from "@webywz/context-diet";

const { text, input, output, cuts } = diet(raw, {
  maxTokens: 2000,
  preset: "logs",
  keep: ["code"],
  redact: true,
});

await dietAsync(raw, { deepen: true });
estimateTokens(text);

Exit codes

| Code | Meaning | | --- | --- | | 0 | Success | | 1 | Usage / missing input / unreadable file / --fail-over-tokens exceeded / --tap not-ok | | 2 | --deepen failed (no key, HTTP error) | | 3 | --max-files limit exceeded |

Develop

npm test && npm run build
node dist/cli.js --demo --preset logs --max-tokens 400

Links

| | | | --- | --- | | npm | npmjs.com/package/@webywz/context-diet | | Issues | github.com/webywz/context-diet/issues | | License | MIT © webywz |


Multi-file budget + SARIF (0.9 / 0.10)

Documented in Features and Quick start above. Summary:

context-diet a.md b.md notes/ --max-tokens 2000 --fail-over-tokens 2500 --json
context-diet --demo --fail-over-tokens 1 --sarif over.sarif
  • Multiple files/dirs share one --max-tokens / --fail-over-tokens budget (JSON budget.shared, budget.maxTokens, sources[]).
  • Directories expand to top-level files unless -R / --recursive is set. --max-files applies after that selection (exit 3). Directory symlinks are not followed.
  • --fail-over-tokens gates the combined dieted output, not each file alone.
  • --sarif <path> writes SARIF 2.1.0 when output exceeds --fail-over-tokens.

中文

多文件共享预算 + SARIF(0.9 / 0.10)

详见上文 功能 与 快速开始。可传多个文件/目录,共享 --max-tokens / --fail-over-tokens(闸门看合并后输出);JSON 含 budget.shared、sources[]。--sarif <path> 在超 --fail-over-tokens 时写出 SARIF 2.1.0。目录默认只展开顶层文件;-R / --recursive 才包含嵌套文件,且不跟随目录符号链接。

演示

npx @webywz/context-diet --demo --preset logs --max-tokens 400

包内自带 examples/noisy.txt(随 npm 包发布):瘦身后的正文在 stdout,人类可读报告在 stderr。无需 clone。

功能

| | | | --- | --- | | 离线启发式 | 剥 ANSI/HTML、去样板、去重、省略 base64 大块 | | 预设 | logs / email / chat / html / code / json | | 预算 | --max-tokens 优先丢掉低价值块(多文件/目录共享);--keep 钉住标题/代码/列表 | | 密钥脱敏 | --redact 擦除 PEM / JWT / sk- / ghp_ / Slack / AWS / Bearer / Basic → [redacted] | | CI token 闸门 | --fail-over-tokens <n>:合并后输出 token 仍 > n 时退出码 1(多文件共享预算;不静默截断);--sarif <path> 写 Code Scanning;--tap 输出 TAP 13;JSON budget 供 Actions | | 管道友好 | 正文 → stdout;报告 → stderr | | 示例随包 | --demo / examples/noisy.txt 在发布 tarball 里 | | 可选 deepen | --deepen 仅在有 key 且你主动开启时走 LLM | | HTML 报告 | --html <path>:写出自包含前后对比报告(离线、无 CDN) | | 多文件 / 目录 | 可传多个文件或目录,共享一份 --max-tokens / --fail-over-tokens(budget.shared、sources[])。目录默认只扫顶层,-R / --recursive 才进入嵌套 | | Step Summary | --github-summary:把精简饮食报告追加到 $GITHUB_STEP_SUMMARY |

安装

npm i -g @webywz/context-diet

需要 Node 18+。无 tokenizer 文件、无 GPU、无账号。

快速开始

context-diet input.txt
npx @webywz/context-diet --demo --preset logs --max-tokens 400
cat noisy.log | context-diet --preset logs --max-tokens 1500 > slim.txt
context-diet notes.md -o slim.md --keep headings,code,lists
context-diet input.txt --dry-run --diff
context-diet --demo --html diet.html -q
context-diet input.txt --json | jq '{tokens: .output.tokens, cuts}'
context-diet dump.txt --redact --fail-over-tokens 2000
context-diet a.md b.md notes/ --max-tokens 2000 --fail-over-tokens 2500 --json
context-diet --demo --fail-over-tokens 1 --sarif over.sarif
context-diet examples/noisy.txt --keep headings|code|lists

预设

| 预设 | 默认保留 | 额外剥离 | | --- | --- | --- | | logs | code | 截断时优先保留 ERROR / WARN / FATAL | | email | headings, lists | 引用回复(> / On … wrote: / Original Message) | | chat | code, lists | 表情反应、进出房系统行、孤立时间戳 | | html | headings, code, lists | <script> / <style> / 注释 / 标签 → 纯文本 | | code | code | 长注释 / 多余空行;截断时优先签名行 | | json | code | 去 JSONC 注释;丢 null / "";稳定 pretty-compact |

显式 --keep 始终覆盖预设默认。

做了什么

除 --deepen 外全部离线。

| 步骤 | 行为 | | --- | --- | | 密钥脱敏 | 可选 --redact,在其他步骤之前 | | 剥 ANSI | 终端颜色 / OSC | | 剥 HTML | 可选 / --preset html | | 省略大块 | 长 base64 / data-URI → [omitted blob: N chars] | | 折叠空白 | 代码围栏内不动 | | 去样板 | 重复日志行、签名、DEBUG 噪声 | | 去引用 / 聊天 chrome | 邮件引用;反应 / 进出房 | | 段落去重 | 精确与近似(Jaccard ≥ 0.84) | | 截断 | --max-tokens + --keep / 预设 | | Token 闸门 | --fail-over-tokens:饮食后对合并输出 token 的硬失败 CI 闸(多输入共享) | | TAP | --tap:TAP version 13(超过 --fail-over-tokens 则退出 1) | | JSON budget | --json 中的 budget.failOverTokens / budget.outputTokens / budget.ok / budget.shared / sources[] |

Token 估算为 cl100k 近似(约 4 字符/词,1 个 CJK ≈ 1 token)。够用来做预算,不是计费 tokenizer。

命令行要点

| 参数 | 含义 | | --- | --- | | [file\|dir ...] | 一个或多个路径;目录默认展开顶层文件(--recursive 才包含嵌套);省略或 - 读 stdin。多输入共享一份 token 预算 | | -o, --output <file> | 把瘦身正文写到文件 | | --html <path> | 写入自包含 HTML 报告(统计 + 截断预览) | | --max-tokens <n> | 估算输出 token 上限(会截断;多文件/目录共享) | | --fail-over-tokens <n> | 合并后输出 token 仍 > n 时退出 1(CI 闸;共享;与截断不同) | | --sarif <path> | 超过 --fail-over-tokens 时写出 SARIF 2.1.0(GitHub Code Scanning) | | --tap | 向 stdout 输出 TAP version 13(超过 --fail-over-tokens 则退出 1) | | --redact | 擦除常见密钥形态 → [redacted](在饮食启发式之前) | | --keep <cats> | 截断时优先保留 headings / code / lists | | --preset <name> | logs | email | chat | html | code | json | | --exclude <glob> | 跳过匹配 glob 模式的文件(可重复;仅目录) | | --include <glob> | 仅保留匹配至少一个 glob 的路径(可重复;仅目录) | | --max-files <n> | 当目录扫描过滤后将处理 > n 个文件时失败(退出码 3) | | -R, --recursive | 同时纳入嵌套目录中的文件。默认仍只扫描顶层。不跟随目录符号链接 | | --demo | 对包内 examples/noisy.txt 瘦身 | | --json | stdout 输出 JSON 报告 | | --dry-run | 只报告,不写正文 | | --diff | stderr 显示被删行(有上限) | | --deepen | 可选 LLM 二次压缩 | | --provider openai\|anthropic | 强制 deepen 提供商 | | --github-summary [path] | 追加饮食 Markdown 到 $GITHUB_STEP_SUMMARY 或指定路径 | | -q, --quiet | 不打印人类报告 |

默认 I/O 适合管道:

pbpaste | context-diet --preset chat --max-tokens 800 | pbcopy

目录扫描

目录默认只展开顶层文件。传入 -R 或 --recursive 后,才会纳入嵌套目录中的文件。递归不会跟随目录符号链接,因此不会因链接成环,也不会顺着链接走出当前目录树。每一层以 . 开头的名字都会跳过。

扫描目录时,context-diet 自动跳过:

  • node_modules/ 和 .git/(始终)
  • .gitignore 中的模式(在项目根或最近的 git 根找到时)
  • 通过 --exclude <glob> 自定义模式(可重复)
  • 以 . 开头的隐藏名字

也可以用 include 模式过滤(仅保留匹配至少一个 --include glob 的文件)。glob 匹配的是相对于所给目录的路径,* 不跨越 /。--exclude '*.tmp' 还会用该模式匹配任意单个路径段,因此会跳过嵌套的 *.tmp。--include '*.ts' 只匹配顶层的 *.ts;在 --recursive 下要用 **/*.ts 才能选中嵌套的 TypeScript 文件。

context-diet src/ --include '*.ts' --include '*.js'
context-diet src/ --exclude '*.tmp' --exclude 'test-*'
context-diet . --max-tokens 2000
context-diet src/ --recursive --include '**/*.ts' --max-files 100

为安全起见,使用 --max-files <n> 防止处理过多文件。计数发生在 include、exclude、gitignore 以及(若开启)递归展开之后:

context-diet docs/ --max-files 100              # 顶层文件 > 100 时退出码 3
context-diet docs/ --recursive --max-files 100  # 嵌套展开后 > 100 时退出码 3

可选 LLM 二次压缩

--deepen 不是必须的。有 key 时:

export OPENAI_API_KEY=sk-...          # 或 ANTHROPIC_API_KEY
context-diet chapter.md --deepen --max-tokens 1200

默认模型:gpt-4o-mini / claude-3-5-haiku-20241022。若模型返回更长,保留本地结果。无 key 时 --deepen 退出码 2,不会联网。

退出码

| 码 | 含义 | | --- | --- | | 0 | 成功 | | 1 | 用法错误 / 缺输入 / 文件不可读 / --fail-over-tokens 超限 / --tap not-ok | | 2 | --deepen 失败(无 key、HTTP 错误) | | 3 | --max-files 超限 |

链接

| | | | --- | --- | | npm | npmjs.com/package/@webywz/context-diet | | Issues | github.com/webywz/context-diet/issues | | License | MIT © webywz |