npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@webywz/llm-roast

v0.1.20

Published

Roast your LLM prompts and answers — fluff, hedges, hallucination vibes, scored 0–100

Readme

llm-roast

A tiny CLI for eval dumps, prompt reviews, and screenshots you can share. Higher Roast Score means more roastable. This is a heuristic scorer, not a fact checker.

给评测 dump、prompt 复盘和可分享截图用的小 CLI。Roast Score 越高越该被吐槽。这是启发式打分,不是事实核查器。


Install · 安装

npm i -g @webywz/llm-roast
npx @webywz/llm-roast examples/answer.md

Requires Node.js 18.18+. Package: @webywz/llm-roast. Binary: llm-roast.

需要 Node.js 18.18+。包名 @webywz/llm-roast,命令仍是 llm-roast。


Quick start · 快速开始

llm-roast prompt.txt
llm-roast answer.md
llm-roast --prompt p.txt --answer a.txt
cat answer.md | llm-roast
llm-roast - < prompt.txt
llm-roast --json answer.md
llm-roast --html --offline answer.md
llm-roast --html --theme fog --offline answer.md
llm-roast --html report.html --theme midnight --offline answer.md
llm-roast --html --theme aurora --offline answer.md
llm-roast --list-themes
llm-roast --list-themes --json
llm-roast --out report.txt answer.md
llm-roast --spicy answer.md
llm-roast --offline --fail-above 60 answer.md
llm-roast --suite "examples/*.md" --json
llm-roast --suite examples --json --fail-above 60
llm-roast --github --offline answer.md
llm-roast --github --quiet --offline answer.md
llm-roast --github --max-annotations 10 --offline answer.md
llm-roast --locale zh --offline answer.md
llm-roast --copy --offline answer.md

| Flag | What it does · 作用 | | --- | --- | | --prompt <file> | Force prompt-smell rules (- = stdin) · 按 prompt 气味规则打分 | | --answer <file> | Force answer rules (- = stdin) · 按回答规则打分 | | --suite <glob\|dir> | Batch mode: roast ≥2 files from glob/directory (JSON output) · 批处理模式:对 glob/目录中 ≥2 个文件打分(JSON 输出) | | --out <file> | Write report to file · 将报告写入文件 | | --html [path] | Self-contained HTML roast report (stdout, or write to path; offline, no CDN) · 自包含 HTML 吐槽报告(stdout 或写入路径;离线、无 CDN) | | --theme <name> | HTML palette only: ember (default, 0.1.16 look), fog, midnight, paper, aurora · 仅 HTML 配色:ember(默认,与 0.1.16 相同)、fog、midnight、paper、aurora | | --list-themes | Print available HTML theme names (one per line; --json: JSON array) · 打印可用 HTML 主题名(每行一个;加 --json 则输出 JSON 数组) | | --json | Machine-readable result · 机器可读输出 | | --spicy | Harsher tone; score unchanged · 文案更狠;分数不变 | | --offline | Skip the optional API encore · 跳过可选 API 加戏 | | --fail-above <n> | Exit 1 when Roast Score > n (suite: maxScore > n; 0–100) · 分数 大于 n 时退出码 1(suite: maxScore 大于 n) | | --locale <en\|zh> | Terminal output locale (default en; JSON keys unchanged) · 终端输出语言(默认 en;JSON 键名不变) | | --github | Emit GitHub Actions ::notice / ::warning / ::error annotations · 输出 Actions 注解,PR Checks 可见 | | --max-annotations <n> | Cap annotation lines with --github (default 20) · 限制 --github 注解条数(默认 20) | | --quiet | Suppress the pretty terminal report; --json / --github still emit · 抑制漂亮终端报告;--json / --github 仍照常输出 | | --copy | Copy the pretty roast report to the clipboard; skip silently if unavailable · 复制漂亮终端报告到剪贴板;无剪贴板工具则静默跳过 | | --color / --no-color | Force / disable ANSI color · 强制 / 关闭颜色 | | -h / -v | Help / version · 帮助 / 版本 |

A bare file is auto-classified from the name (prompt, answer, response, …) or opening lines (you are vs as an AI).

单文件会按文件名或开头几行自动判断是 prompt 还是 answer。

Suite / batch mode · 批处理模式

Roast multiple files in one run with --suite <glob|dir>:

一次对多个文件打分,使用 --suite <glob|dir>:

llm-roast --suite "examples/*.md" --json
llm-roast --suite examples --json
llm-roast --suite "examples/*.{txt,md}" --json --fail-above 60
llm-roast --suite "examples/{answer,bad-answer}.md" --json
llm-roast --suite "fixtures/{good,bad}/*.md" --json

Suite mode requires ≥2 roastable files (.txt, .md, .markdown). Supports brace expansion like {a,b} for flexible file matching. Output is always JSON with per-file results and aggregated stats:

批处理模式需要 ≥2 个可打分文件(.txt、.md、.markdown)。支持 大括号扩展 如 {a,b} 以灵活匹配文件。输出始终为 JSON,包含每个文件的结果与汇总统计:

{
  "version": "0.1.20",
  "files": [
    { "path": "examples/answer.md", "score": 73, "label": "yikes", "kind": "answer", "wordCount": 89 },
    { "path": "examples/prompt.txt", "score": 58, "label": "caution", "kind": "prompt", "wordCount": 42 }
  ],
  "stats": {
    "count": 2,
    "maxScore": 73,
    "avgScore": 66,
    "failCount": 0
  }
}

--fail-above in suite mode applies to stats.maxScore (the highest score across all files). Exit code 1 when maxScore > threshold.

批处理模式下的 --fail-above 应用于 stats.maxScore(所有文件中的最高分)。当 maxScore > 阈值 时退出码为 1。

Exit codes · 退出码:

  • 0 — Success · 成功
  • 1 — --fail-above threshold exceeded · 超过 --fail-above 阈值
  • 2 — Usage error or I/O failure · 用法错误或 I/O 失败

Fixtures: examples/prompt.txt · examples/answer.md.

示例:examples/prompt.txt 与 examples/answer.md。

# GitHub Actions — annotations + CI gate + job summary
- name: Roast gate
  run: npx @webywz/llm-roast --offline --github --quiet --fail-above 60 --max-annotations 20 path/to/answer.md

Copy-paste workflow: examples/github-actions.yml.

可直接复制的 workflow:examples/github-actions.yml。

--github prints workflow commands (::warning file=…::…) so Checks show score and findings. High severity → error, medium → warning, low → notice; score bands use the same scale. With GITHUB_OUTPUT set it also writes score, label, findings. With GITHUB_STEP_SUMMARY set it appends a short Markdown job summary (score / label / key findings). Use --max-annotations <n> to soft-cap PR noise (default 20). --quiet suppresses the pretty terminal report only (--json / --github annotations, summary, and outputs still emit). Combine with --json and annotations go to stderr so stdout stays parseable. Works together with --fail-above (annotations still emit, then exit 1 when over the gate).

--github 输出 workflow 命令,PR Checks 能直接看到分数与 findings。高严重 → error,中 → warning,低 → notice;总分按档位映射。在 Actions 里还会写 GITHUB_OUTPUT(score / label / findings),有 GITHUB_STEP_SUMMARY 时再追加一段简洁 Job Summary。--max-annotations <n> 可限制注解条数(默认 20)。--quiet 只抑制漂亮终端报告(--json / --github 注解、summary、outputs 仍照常)。与 --json 联用时注解走 stderr,stdout 仍是干净 JSON。可与 --fail-above 组合:先出注解,超阈再以 1 退出。

--fail-above 100 never fails (score tops at 100). --fail-above 0 fails on any non-zero score.

--fail-above 100 永远不会失败(满分 100);--fail-above 0 只要非零就失败。

Library · 库

import { roastText, roastDocuments } from "@webywz/llm-roast";

const answer = roastText(src, { kind: "answer", spicy: false });
console.log(answer.score, answer.roast, answer.findings);

const both = roastDocuments([
  { text: prompt, kind: "prompt", source: "prompt.txt" },
  { text: reply, kind: "answer", source: "answer.md" },
]);

Features · 特性

| Bucket · 桶 | Examples · 例子 | | --- | --- | | Fluff | "as an AI language model", "in today's digital age", "delve", "tapestry", "robust", "leverage", "synergy", "great question" | | Hedge | "might", "could potentially", "it depends", "perhaps", "to some extent" | | Overconfidence | "always", "never", "guaranteed", "100%", "everyone knows" — extra weight when no nearby source · 附近没有出处时更重 | | Hallucination vibes | (Author et al., 2024) with no references, example.com docs, precise % with no source, invented APIs, "studies show" | | Prompt smells | concise and comprehensive, no lists and bullet the points, stacked roles, missing format / audience / length |

  • Deterministic copy — same input → same roast text. --spicy only changes tone. 文案确定 — 相同输入得到相同吐槽;--spicy 只改语气。
  • CI gate — --fail-above still prints the roast, then exits 1 only when the score exceeds the threshold. CI 门禁 — 仍会打印吐槽;仅当分数超过阈值时以 1 退出。
  • GitHub annotations — --github emits Actions notices/warnings/errors for PR UI; optional Job Summary + --max-annotations; --quiet for annotations-only CI logs. GitHub 注解 — --github 输出 Actions notice/warning/error;可选 Job Summary 与 --max-annotations;--quiet 只留注解不打漂亮报告。
  • HTML export — --html emits a self-contained offline roast (score, findings, spicy/summary). Omit the path for stdout; give a path to write the file (parent dirs created). --theme <name> swaps CSS-variable palettes (ember default = 0.1.16 look, plus fog, midnight, paper, aurora); it does not change JSON, pretty, GitHub annotations, or scoring. --list-themes prints those names (one per line; --list-themes --json is a JSON array). --html and --json cannot share stdout. HTML 导出 — --html 写出离线自包含吐槽报告(分数、findings、spicy/summary)。省略路径打印到 stdout;给出路径则写文件(自动创建父目录)。--theme <name> 只换 CSS 变量配色(默认 ember 与 0.1.16 相同,另有 fog / midnight / paper / aurora);不影响 JSON、终端漂亮输出、GitHub 注解或打分。--list-themes 打印这些主题名(每行一个;--list-themes --json 为 JSON 数组)。--html 与 --json 不能同时占用 stdout。
  • Clipboard — --copy writes the pretty terminal roast (not JSON / HTML / GitHub chrome) to the system clipboard. Missing pbcopy / wl-copy / xclip / xsel / clip → skip silently. 剪贴板 — --copy 把漂亮终端吐槽报告(不是 JSON / HTML / GitHub 装饰)写入系统剪贴板。没有 pbcopy / wl-copy / xclip / xsel / clip 则静默跳过。
  • Optional deepen — with OPENAI_API_KEY (preferred) or ANTHROPIC_API_KEY, one extra paragraph after the heuristic roast. Failures and timeouts stay quiet. --offline skips it. 可选加深 — 有 OpenAI(优先)或 Anthropic Key 时,启发式之后多一段评语;失败 / 超时静默。--offline 跳过。

Links · 链接

npm install && npm test && npm run typecheck