jev-lint
v0.9.1
Published
A natural-language linter: ast-grep picks the code, a sentence you write is the rule, and Jev returns the verdict
Maintainers
Readme
jev-lint
A lint tool that uses Jev -- a fast classifier that answers a natural-language question with a calibrated probability instead of writing text -- to decide what a parser cannot.
Try it on tracked files
Try it in a Git working tree with Node 24+. Set OPENROUTER_API_KEY to your OpenRouter key, then run:
npx -y jev-lint --model openrouter:perplexity/pplx-decider-v1.3:freeWithout a config, the CLI asks once, selects starter rules from git ls-files, prints the proposed YAML, then lints a small sample. Both rule selection and lint use the free Decider. Selection sees up to 64 tracked filenames (8 KiB of paths). The check reads up to three eligible files, at most 16 KiB each and 32 KiB combined. It leaves saving the YAML to you. Run npx -y jev-lint --dry-run to inspect the inventory without a key.
For ongoing use and CI, we strongly recommend Jev with TYPESAFE_API_KEY. The shipped thresholds are calibrated on Jev. Set your TypeSafe key and run npx -y jev-lint. If both keys are exported, unset OPENROUTER_API_KEY to use TypeSafe directly. An existing config uses its normal file and rule selection.
Example finding
This function is wrong, and nothing in your toolchain will say so:
/** Returns the cart, or null when no cart has this id. */
export function getCart(id: string): Cart {
const cart = carts.get(id);
if (!cart) throw new Error(`no cart ${id}`);
return cart;
}The comment promises null. The body throws. The return type is Cart, not
Cart | null. It type-checks and it lints clean.
A jev-lint rule is one sentence. An ast-grep matcher picks which code to look at, and a model is asked that sentence about each match -- here, the comment above this code claims something that is not true of the code -- answering with a probability. The code being judged is sent to an API, which is what the key pays for; nothing runs locally except the matcher.
export TYPESAFE_API_KEY=... # https://typesafe.ai
npx -y jev-lint check examplesexamples/cart.ts
16 flag The comment above this code claims something that is not true of the code.
comment-describes-declaration 0.89 cutoff 0.56
...0.89 is the model's agreement and 0.56 the cutoff this rule ships with; under it, nothing is reported. Six findings over two files, for a tenth of a cent; what a bigger run costs is measured, not extrapolated.
What it catches
Sixty-eight distinct rule IDs ship, most of them asked in several languages -- 121 rules in all. Roughly:
- A name against the thing it names. A function, method, binding, type,
class, trait or module; an npm script against the command it runs; an sqlc
query name against its SQL.
applyDiscountthat also saves the cart,isAdminbound to a string. - A guarantee the name implies. Whether
safe*is safe,pure*is pure, an idempotent-sounding function is idempotent, whether Go'sMustXpanics on the failure its name promises. - A comment against the code under it. The doc comment above a
declaration, a comment inside a block, and a stated failure contract --
@throws,# Errors,Raises:-- against the body. - A test against what it claims. A test that would pass with the behaviour its name claims broken; a failure path no test reaches; a snapshot standing in for a behaviour claim; a mock that has replaced the subject.
- A failure that goes quiet. A
catchthat hides one, an error message that does not match its condition, a log level that does not match the event. - What a shell script does to the machine that its reader cannot see: run code it downloaded, read secrets it was not given, install persistence, delete past its own scope, weaken a defence, open a way in, take orders from elsewhere, hide what it runs.
- Whether a document is worth reading. Slop, filler, padding, vagueness; a section that opens with an agenda or ends by previewing the next.
- Whether
AGENTS.mdis usable. Each section's instructions must be clear enough to follow and consistent with the rest of the file. - A commit message against its diff, and a change against the
instructions the repository wrote for itself --
AGENTS.mdin its own tree, orCLAUDE.mdwhenAGENTS.mdis absent, read as of that change. Only the instructions a diff can be held against: "never edit the generated file" is judged, "develop test-first" is not.
RULES.md is all of them, each with its cutoff and its score on its own fixtures. What makes them one family is that each holds a claim the code makes about itself against what the code does -- which is visible only to a reader who has read both.
It is not for anything a compiler, type checker or ESLint already
decides. This kind of model is good at code that contradicts itself and poor
at defects needing knowledge of a specific API, such as .sort() defaulting
to lexicographic order -- measured, with the evidence.
Keep your existing tools for those.
Install
For a project dependency:
npm install --save-dev jev-lint
npx jev-lint init # writes .jev-lint.yaml
npx jev-lint check src --dry-run # what it would ask, and the price. No request.The key is read from the environment only — OPENROUTER_API_KEY, TYPESAFE_API_KEY, or
TYPESAFEAI_API_KEY — and never from the config file, which belongs in
version control and a key does not. Everything that does not ask the model
works without one: --dry-run prices a run, jev-lint rules lists what
would run, jev-lint replay re-scores a recorded run, and CI can lint from
a committed cache with no key at all.
Model recommendation
The default is pinned to jev-1.14.0, the
current stable release, so a moving alias cannot
silently change answers at fitted cutoffs. Override it with --model, config's
model:, or JEV_LINT_MODEL. --model jev-latest follows future stable releases.
Jev is the recommended model for normal linting. The shipped rules and cutoffs are calibrated for Jev, and it performed best at those cutoffs in the 81-suite fixture comparison.
When the CLI detects OPENROUTER_API_KEY, it asks before sending data to OpenRouter for
typesafe/jev-1.14, even if a TypeSafe key is also present. Enter cancels the run.
For CI, set model: openrouter:typesafe/jev-1.14 in .jev-lint.yaml, pass
--model openrouter:typesafe/jev-1.14, or set JEV_LINT_MODEL=openrouter:typesafe/jev-1.14.
For default Jev, apiKeyEnv: OPENROUTER_API_KEY in the config also selects OpenRouter.
Without an explicit selection, a noninteractive run exits 2. Unset the OpenRouter key to use TypeSafe directly.
Experimental OpenRouter decision models are selectable with
--model openrouter:<id> and OPENROUTER_API_KEY. Supported IDs are
liquid/d1.2-ultra, liquid/d1.2, baseten/mammoth-1, and
perplexity/pplx-decider-v1.3:free. In the
November comparison, Perplexity's free model
matched Jev's aggregate results on 146 naming cases, but selected too many
starter rules. The free trial therefore offers the model a shortlist of core rules. Other OpenRouter checking models use Jev for starter selection.
export OPENROUTER_API_KEY='<your-api-key>'
npx jev-lint run typescript/fn-name-promises src \
--model openrouter:perplexity/pplx-decider-v1.3:freeThese are opt-in experiments with existing cutoffs, not replacements validated
across the catalog. Live OpenRouter cost comes from billed usage.cost;
free requests are paced at 3.2-second intervals and remain subject to daily limits.
Native models get separate verdict-cache files; use --cache none for fresh comparisons.
Cloudflare Clef and Clef-flash are optional experimental models for comparison
from a repository checkout. just clef-plan previews the requests and cost;
just clef-eval runs the comparison with Cloudflare credentials. See the
evaluation guide for setup, recorded results and
the limits of fitting new cutoffs on the same fixtures.
Using it
| | |
| --- | --- |
| jev-lint / jev-lint auto src | confirm, then use existing settings or select starter rules and check; --yes skips the startup prompt |
| jev-lint check src | judge whole files |
| jev-lint review --base main | only the lines a diff touched — where findings concentrate, at a fraction of the cost |
| jev-lint commits --base main | each commit's message against its diff, and each change against the AGENTS.md the repository wrote for itself |
| jev-lint commits --staged | the staged change against repository instructions; there is no commit message yet |
| jev-lint run typescript/fn-name-promises src | one rule; --file mine.yml for one of your own |
| jev-lint rules | every loaded rule: its question, cutoff and file |
| jev-lint introspect --write | ask Jev to select starter rules from directory metadata and create a new config |
| jev-lint init --introspect | initialize with the same recommended starter; --dry-run only plans |
introspect --dry-run shows the metadata without requests. introspect --write
uses Jev directly, with no separate LLM or agent, and creates a config containing
only recommended starter rules. Existing configs are preserved; use
--write --output .jev-lint.proposed.yaml for a separate proposal.
The report lists enabled, deferred and unsuitable rules with representative paths.
Source bodies, script commands and environment values are not sent.
See the initialization guide.
init --introspect is equivalent to introspect --write.
init --introspect --dry-run makes no requests and writes no configuration.
The config picks the rules, as ESLint's does. init writes every shipped
rule turned on, as a catalogue to prune:
files: [src, test]
exclude: [test/fixtures]
rules:
typescript/fn-name-promises: on
rust/fn-name-promises: off
typescript/comment-describes-block: { threshold: 0.7, severity: error }An answer at or above threshold becomes a finding. The old at spelling
still loads with a warning. hooks.precommit can inherit this rules: map or
select its own rules for staged review; see the hook guide.
Use language/id for namespaced rules. A bare shipped id still selects every
language with that id, but jev-lint warns because it can enable a language
unintentionally. A flat project rule has no namespace and does not warn.
To silence one finding, or one file, in any comment syntax:
// jev-lint-ignore-next-line fn-name-promises
// jev-lint-ignore-fileA suppressed subject is never sent, so a suppression also saves its tokens.
In CI and git hooks — review --base "$GITHUB_BASE_REF" --format github
annotates a pull request; jev-lint init --pre-commit and --pre-push write
hooks. Neither blocks until you raise a rule to severity: error, and no
shipped rule has it. See docs/use-hooks.md, which also
covers the exit code that will stop you committing while offline.
What ships
121 rules under rules/<language>/<id>/, and the files you point it at decide
which run. Nothing to select.
| point it at | what runs |
| --- | --- |
| .ts .tsx .js .jsx | typescript/ — 23 rules, first tier |
| .rs | rust/ — 9, first tier |
| .py .go | python/ — 14, go/ — 9 |
| .mbt | moonbit/ — 20, once the parser is declared |
| .vibe | vibe/ — 2, with the vibe-lang parser |
| .sh .bash .zsh | shell/ — 8: what a script does to the machine that its reader cannot see |
| .md .mdx | markdown/ — 13: 11 writing rules and 2 agent-instruction rules |
| package.json | json/ — 1 |
| sqlc queries and prepared architecture evidence | text/ — 17 total, including 16 design advisories |
| commits | git/ — 4 including 2 metric advisories |
Public API review uses three units: an exported name, the signatures at one entry point, and one documented workflow across import paths. Rule selection and layout example.
typescript and rust are first tier: every rule under them carries
fixtures, expectations and an accepted baseline. The rest are calibrated to
the same bar and not yet promised. RULES.md lists all 121 with
their question, cutoff and score on their own fixtures.
Your own rule can target any language ast-grep parses — Java, Kotlin, Swift, C, C++, C#, Ruby, PHP, Lua, Dart, Scala, Elixir, Haskell, Solidity, HTML, CSS, YAML — and any other grammar you compile.
It is not deterministic, and not always right
Measured on this repository, about one finding in five was wrong, and a score
near a cutoff can move by a few hundredths between runs. jev-lint is built
around that rather than pretending otherwise: verdicts are cached by the
content of the rule and the code, --retry 3 decides on the mean of
three passes and marks what did not reproduce, --loose lists the band under
a cutoff for a reader without making it a finding, and cutoffs are yours to
fit — the shipped ones are fitted to this package's corpus, not yours.
Read a finding as a candidate for a human to judge, not a verdict to act on.
Reviewed code, comments, prose, and commit messages can contain instructions that manipulate model judgments, including answers with a valid response schema. Keep security-sensitive decisions behind independent checks and human review. SECURITY.md describes the trust boundaries, safeguards, and limits of response validation.
Writing a rule
An ast-grep matcher decides which code is looked at; one sentence decides whether it is a problem:
id: catch-hides-failure
languages: [TypeScript, Tsx]
rule: { kind: catch_clause }
subject: enclosing # judge the function around the match, not the clause
ask: This catch block swallows a failure the caller needed to know about.docs/writing-rules.md is the walkthrough, including the fixtures and the fitted cutoff a rule ships with.
For a coding agent
The repository ships a skill — how to run jev-lint and use its packs, plus a separate guide to writing rules in your own repository for code, custom grammars, text files, and Git changes:
Project initialization and rule recommendations
npx skills add mizchi/jev-lint --skill jev-lint--skill jev-lint matters: the repository also carries jev-lint-repo, the
maintainer's skill for changing jev-lint itself. As a Claude Code plugin,
which also installs /jev-lint:review, /jev-lint:commits,
/jev-lint:prose and /jev-lint:new-rule:
/plugin marketplace add mizchi/jev-lint
/plugin install jev-lint@jev-lintskills/jev-lint/ is the single source for the user-facing skill.
.claude/skills/jev-lint is a tracked symlink to it; edits appear in both
places immediately. .claude/skills/jev-lint-repo is a separate maintainer
skill. Run just skills-check to verify the link or just skills-sync to
restore it. CI checks the link so a copied skill cannot drift silently.
Documentation consistency
jev-lint docs compares explicit claims, glossary definitions and translations,
retaining both source locations. This repository checks its own README pair,
API guides and contributor instructions against generated project facts and
canonical terminology.
jev-lint docs --dry-run
jev-lint docs --record .jev-lint/documents-baseline.json
jev-lint docs --replay .jev-lint/documents-baseline.jsonThe manifest is .jev-lint/documents.json; --document-manifest selects another.
Replay is offline and rejects changed sources or questions. Contradictions exit 1;
incomplete or stale assessments exit 3. The report remains UNREVIEWED.
See the self-check guide for scope, budgets and evaluation.
Documentation
| | | | --- | --- | | RULES.md | every shipped rule, its cutoff and its fit | | docs/reference.md | every flag and field, calibration, batching, limits | | docs/audit-api.md | versioned screening results, pinned policy, strict cache, request controls and adversarial evaluation | | docs/relations-api.md | approved specification sources, multi-source comparisons, claim grounding and drift coverage | | docs/relations-workflow.md | comparison discovery, bounded dependency context, workflow replay and consumer integration | | docs/adoption.md | accepted findings, file eligibility, doctor, suppression diagnostics and SARIF export | | docs/self-check.md | semantic consistency, terminology, translation and snapshot replay | | docs/project-facts.md | generated current catalog, runtime, default model, CI and package facts | | docs/glossary.md | canonical terms and Japanese aliases | | docs/use-hooks.md | git hooks and CI | | SECURITY.md | model judgment risks, trust boundaries, and safeguards | | docs/writing-rules.md | writing a rule, and the evals it ships with | | docs/cost.md | what a full run costs, measured | | docs/deepdive.md | everything measured that is still true, and the evidence | | docs/internal/architecture.md | how the code works, for changing it | | docs/internal/findings.md | the notebook, in the order it happened, wrong turns included | | CHANGELOG.md | every release | | docs/releasing.md | version checks, installed-package validation and tag-triggered OIDC publishing |
Prior art
The idea of asking a model a lint question comes from
mizchi/jev-playground's
eslint-plugin-jev. What is different here: relational multi-language
matchers instead of single-node ESLint selectors, a custom runner instead of
ESLint's synchronous per-file callback, diff-scoped review mode, the state
arm as a measured axis rather than a fixed choice, and record/replay so a
threshold is auditable.
Two sibling projects on the same model shaped parts of this one, and are worth reading beside it:
- devagrawal09/jev-review — a
review workflow that screens whole patches and follows the strongest
signals through
choiceandscorequestions. Its structured criteria, its screen-then-classify shape (--explain,--loose), its test-gap screen (thepairedarm andtests-cover-failure-paths) and its diff subjects (jev-lint commits) were taken from it and measured here;docs/internal/findings.md§14 says what each measured as. - TKY-27/JevSlop (MIT) — an AI Slop
Score for note.com articles: eight writing-quality signals and one
whole-article judgment, each a five-level rubric. The nine rules under
rules/markdown/are those rubrics, verbatim or reversed so that a high score always means the defect the rule names, applied to a Markdown file as a whole; two rubrics that named amounts rather than defects had to be reworded before they separated, and the rule files say how. As JevSlop says of itself: a writing characteristic, not an authorship probability. - k16shikano's cognitive-rhythm writing norm
— a Japanese norm for explanatory prose whose post-writing checks ask the
question this tool asks of code: does a sentence update the subject, or
only the document? Three rules under
rules/markdown/are those checks (section-ends-with-a-preview,section-opens-with-an-agenda,document-abandons-a-question), two more are candidates with the reason they are not shipped, and/jev-lint:proseruns them beside the norm's mechanical leakage test. They read the norm's genre — articles and chapters — and will flag a reference or a findings log for doing what a reference does.
MIT.
