agent-liedetector-skill
v0.3.2
Published
A confidence-calibration skill for any AI coding agent. Tags every claim π’ [VERIFIED] / π‘ [INFERRED ~80%] / π [GUESSED ~30%] / π΄ [UNKNOWN] so you can scan how much to trust each line.
Maintainers
Readme
npx agent-liedetector-skill installThat's it. Next session, every research- or decision-relevant claim your agent makes will carry a tag like π’ [VERIFIED], π‘ [INFERRED ~80%], π [GUESSED ~30%], or π΄ [UNKNOWN].
the problem.
Large language models confabulate. They sound confident whether they checked the source or made it up. By the time you notice the bug they introduced, you've already shipped it.
The conventional fixes don't work:
- Asking "are you sure?" β they just rephrase the same wrong answer with more hedging
- "Cite your sources" β they fabricate plausible-looking citations
- "Be honest about uncertainty" β they bury it inside confident prose
the fix.
A 7-tag confidence protocol baked into the agent's system prompt. Every claim that matters gets a tag based on how the agent came to believe it:
| Tier | Tag | Meaning | What you do |
|---|---|---|---|
| π’ | [VERIFIED] | Checked the source firsthand this session | Trust it |
| π’ | [KNOWN] | Well-documented public fact (RFC, spec, mainstream API) | Trust it |
| π‘ | [INFERRED] | Logical deduction from verified premises | Spot-check |
| π‘ | [ASSUMED] | Taken as true to make progress | Override if it matters |
| π | [GUESSED] | Pattern-match guess, no direct evidence | Treat as hypothesis |
| π | [STALE] | True at training cutoff, may have moved | Re-check current docs |
| π΄ | [UNKNOWN] | Outside reliable knowledge | Refused to fabricate. Hand off |
Plus optional decile-snapped calibration on yellow/orange tags: π‘ [INFERRED ~80%], π [GUESSED ~30%]. The ~ signals it's a rough self-estimate, not a calibrated probability.
before & after.
Without agent-liedetector-skill:
The
buildClaudeSettingsfunction deduplicates hooks before writing them to settings.json. The schema'sInheritance merges all three with concat + deduperule guarantees this.
Sounds confident. Is it true? You'd have to read the code to find out.
With agent-liedetector-skill:
π’
[VERIFIED]ThebuildClaudeSettingsfunction reads its own previous output and re-appends hooks β I traced the exact lines. π‘[INFERRED ~80%]The dedupe-by-JSON-signature fix correctly closes the symptom β I didn't run the tests. π[GUESSED ~40%]Other parts ofbaseSettings(mcpServers, plugins) might have similar bugs β pattern-match only, no specific evidence.
Now you know which line to trust, which to verify, and which is hand-waving.
install.
Option 1 β npx (recommended)
npx agent-liedetector-skill installDrops the liedetector skill into ~/.claude/skills/liedetector/SKILL.md (where Claude Code's skill loader picks it up) and offers to append the always-on protocol to ~/.claude/CLAUDE.md so it applies to every session, not just when triggered.
Variants:
npx agent-liedetector-skill install --global # also write to ~/.claude/CLAUDE.md without asking
npx agent-liedetector-skill install --skill-only # only install the skill, don't touch CLAUDE.md
npx agent-liedetector-skill uninstall # remove both
npx agent-liedetector-skill status # check what's installedIdempotent β safe to re-run. The CLAUDE.md modifications are wrapped in <!-- agent-liedetector-skill:start --> ... <!-- agent-liedetector-skill:end --> markers so uninstall is precise.
A short alias is registered too: npx liedetector install does the same thing.
Option 2 β universal (works with 51+ agents)
If you use Cursor, Codex, Cline, Gemini CLI, Aider, GitHub Copilot, Windsurf, Roo, OpenCode, or any of the other 51+ agents in the skills.sh ecosystem, use the universal installer:
npx skills add NagyVikt/agent-liedetector-skillThe skills CLI auto-detects which coding agent you have installed and drops SKILL.md into the right directory for that agent. No need to know paths.
Common variants:
# install to a specific agent (or several at once)
npx skills add NagyVikt/agent-liedetector-skill -a cursor
npx skills add NagyVikt/agent-liedetector-skill -a claude-code -a codex -a cursor
# install to every agent it detects on your machine
npx skills add NagyVikt/agent-liedetector-skill --all
# install globally (~/<agent>/skills/) instead of into the current project
npx skills add NagyVikt/agent-liedetector-skill -g
# CI/CD-friendly, no prompts
npx skills add NagyVikt/agent-liedetector-skill -a claude-code -g -yOption 3 β manual copy
If you don't want to run a script, just copy SKILL.md into your agent's skill directory:
| Agent | Where to copy SKILL.md |
|---|---|
| Claude Code | ~/.claude/skills/liedetector/SKILL.md |
| Codex | ~/.codex/skills/liedetector/SKILL.md |
| Cursor | Paste contents into .cursorrules (whole repo) or .cursor/rules/liedetector.mdc (per repo) |
| Cline | Paste contents into .clinerules |
| Gemini CLI | ~/.gemini/skills/liedetector/SKILL.md |
| GitHub Copilot | Append to .github/copilot-instructions.md |
Option 4 β via cue profile
If you use cue / cuecards, add to any profile:
# profiles/<your-profile>/profile.yaml
persona_includes:
- integrity-protocol # cue ships this snippet built-inhow it works.
The protocol gets injected into the agent's system prompt as a persona snippet. Claude Code reads ~/.claude/CLAUDE.md on every session start, so the rules are in context before you type anything. The skill (~/.claude/skills/liedetector/SKILL.md) is the discoverable reference document β when you ask "what does [ASSUMED] mean?", the agent loads the skill and explains.
No daemon. No background process. No telemetry. Just markdown files in your agent's config directory.
why the calibration % matters.
LLM self-reported probabilities are notoriously miscalibrated as absolute values β ~70% doesn't truly mean "right 7 times out of 10."
But relative ordering across claims in the same response IS meaningful. "I'm ~80% on X, ~50% on Y" tells you which to verify first. That's the actual signal.
To prevent false precision, all percentages snap to deciles (20/30/40/60/80/90) β never ~67% or ~73%. And the leading ~ is non-negotiable so no one mistakes it for a measured statistic.
faq.
No. The protocol adds ~200 tokens to the system prompt β one-time cost per session, imperceptible vs the agent's own context budget.
As an absolute probability, no β LLM self-calibration is unreliable at fine resolution. As a relative ordering across claims in the same response, yes. If the agent tags claim A as ~80% and claim B as ~40%, you can trust that the agent finds A more credible than B. That's the signal worth acting on.
The protocol explicitly forbids grade inflation: "If you didn't actually check, it isn't [VERIFIED]." And when in doubt between two tiers, the rule is to downgrade by default β false confidence hurts more than false hedging. If you notice the agent abusing [VERIFIED], paste the SKILL.md content into the chat and ask it to recalibrate.
4 tags lose distinctions that matter (e.g. "I read the code" vs "training-data fact" are very different epistemic states). 10+ tags become hard to remember. 7 is the sweet spot β split along two axes (how the agent came to believe it Γ how strongly to trust it) without overflowing working memory.
Yes β the protocol is agent-agnostic. Copy SKILL.md content into any LLM system prompt and it works. The bundled install script defaults to Claude Code paths; manual install instructions for Codex, Cursor, Cline, Gemini, and Copilot are above.
"Be honest" is too abstract β the model interprets it as "add hedging language" and otherwise behaves the same. The 7-tag protocol gives it a structured vocabulary for being honest. Each tag has a precise definition; the model knows which to apply because the definitions distinguish them by evidence type, not vibes. Structured uncertainty beats prose hedging.
No. agent-liedetector-skill is a pure markdown injection β it writes two files into your agent's config dir and exits. Nothing leaves your machine.
credits.
The 7-tag protocol was developed inside the cue / cuecards profile manager as a baseline persona snippet (resources/personas/integrity-protocol.md) for AI coding agents. This repo extracts it as a standalone, npx-installable skill so anyone can adopt it without adopting cue.
If you find this useful, also check out:
- cue / cuecards β full agent profile manager with per-directory profiles, skill discovery, and 33 pre-built expert agent loadouts
- claude-mem β persistent cross-session memory for Claude Code
contributing.
git clone https://github.com/NagyVikt/agent-liedetector-skill.git
cd agent-liedetector-skill
node bin/cli.js install --dry-run # see what install would do without writing| Want to | How |
|---|---|
| Suggest a new tag | Open an issue with the proposed name + definition + use case |
| Improve a tag definition | PR against SKILL.md |
| Add an agent adapter | PR against bin/cli.js install path resolution |
| Report a bug | Open an issue |
License: MIT.
