hugcheck

Sort through Hugging Face models and check the trust signals before you pull.

The Hub has over a million models. Search is vibes, and supply-chain risk is real: pickle-format weights can execute arbitrary code on load, typosquat repos exist, and plenty of uploads have no card, no license, and no provenance. hugcheck surfaces the signals so you can decide with your eyes open.

What it is not: a guarantee. hugcheck reports observable signals — it can't prove a model's weights are clean or its publisher is honest. The verdict tiers are a starting point, not a verdict on truth.

Install

npm install -g hugcheck
# or just run it
npx hugcheck audit meta-llama/Llama-3.2-1B

Requires Node 18+. No API key needed — it uses the public Hub API.

Usage

# Full trust report for one model
hugcheck audit Qwen/Qwen2.5-7B-Instruct

# Search, ranked with trust badges (trust first, then downloads)
hugcheck search "7b instruct" --limit 8

# Side-by-side comparison
hugcheck diff Qwen/Qwen2.5-7B-Instruct mistralai/Mistral-7B-Instruct-v0.3

# Machine-readable output for scripts and agents
hugcheck audit <model> --json

audit exits 0 for clear/caution, 2 for avoid (handy for CI gates), 1 on errors.

The checks

| Check | What it looks at | |---|---| | Weight format | safetensors vs pickle (.bin/.pt/.pth/…) in the repo file list. Pickle-only is a fail — deserialization is code execution. CoreML/ONNX export dirs are excluded (their .bin files aren't pickles). | | Publisher | Publisher name vs a curated list of known orgs, with typosquat detection (homoglyph normalization like g00gle→google, plus edit-distance). Exact match passes; typosquat fails; near-miss warns. | | Model card | Is there one with actual metadata, or an empty husk? | | License | Declared in card metadata or tags? | | Maintenance | Last-updated date. Weights don't rot like software, so the bar is generous (2 years). | | Repo files | Custom modeling_*.py / configuration_*.py files auto-execute on load — flagged for review. | | Access | Disabled repos fail; gated repos are noted. |

Tier = worst check outcome: any fail → AVOID, any warn → CAUTION, else CLEAR.

The known-org list (src/util.ts) is curated, not exhaustive — it's the orgs attackers actually squat. PRs welcome if a legit org is missing.

Why no score?

A 0–100 trust score pretends at precision nobody has. A checklist with the evidence shown lets you apply your own judgment — which is the only judgment that counts here.

License

MIT