Will this model run on my machine, and how fast?

The answer currently lives in scattered Reddit comments that die in a week. Every person who downloads a model has the answer sitting on their disk and no place to put it. This is the place, and the collection is one command.

npx github:mohitagw15856/runs-on-what --model qwen3:8b     # detects your hardware, runs, opens a PR
npx github:mohitagw15856/runs-on-what --dry-run            # just show me what you detected

Runs straight from this repo — nothing published to npm yet, nothing to install.

Every measurement

| Model | Device | Decode tok/s | Prefill tok/s | Peak mem | Runtime | Spread | Measured | | --- | --- | --: | --: | --: | --- | --: | --- | | llama3.1:8b | Apple M5 (16 GB) | 24.8 | 991.1 | — | ollama 0.32.13 | ±0.8 | 2026-08-15 | | qwen3:8b | Apple M5 (16 GB) | 21.7 | 931.8 | — | ollama 0.32.13 | ±2.4 | 2026-08-15 | | gemma3:12b | Apple M5 (16 GB) | 15.1 | 282.5 | — | ollama 0.32.13 | ±0.3 | 2026-08-15 | | phi4:14b | Apple M5 (16 GB) | 12.2 | 527.0 | — | ollama 0.32.13 | ±1.1 | 2026-08-15 |

Browse

| By device | | By model | | | --- | --: | --- | --: | | Apple M5 (16 GB) | 4 | Qwen3 8B | 0 | | Apple M4 Max (128 GB) | 0 | Qwen3 14B | 0 | | Apple M4 Pro (48 GB) | 0 | Qwen3 32B | 0 | | Apple M3 Max (128 GB) | 0 | Llama 3.1 8B | 0 | | Apple M2 Max (64 GB) | 0 | Llama 3.3 70B | 0 | | Apple M1 Max (32 GB) | 0 | Gemma 3 12B | 0 | | NVIDIA RTX 5090 (32 GB) | 0 | Gemma 3 27B | 0 | | NVIDIA RTX 4090 (24 GB) | 0 | Mistral Small 24B | 0 | | NVIDIA RTX 3090 (24 GB) | 0 | Phi-4 14B | 0 | | NVIDIA RTX 4060 Ti (16 GB) | 0 | DeepSeek-R1 Distill 14B | 0 | | Raspberry Pi 5 (8 GB) | 0 | | |

Why a script instead of "post your numbers"

Because numbers measured with different prompts, context lengths and sampling settings cannot be compared, and a table of incomparable numbers is worse than an empty one — it looks like an answer. So the harness fixes all of it:

| Setting | Value | | --- | --- | | Prompt | fixed, hashed Explain how a bicycle derailleur shifts gears. C… | | Tokens generated | 256 | | Temperature | 0 | | Context | 4096 | | Repetitions | 3, median reported | | Spread | published, not hidden |

That last one matters. A thermally throttled laptop and a well-cooled one can produce the same median and wildly different spreads. Hiding the spread would let the table lie quietly.

The harness is versioned. When it changes, old rows are labelled, not deleted — throwing away history to keep a table tidy is how a benchmark repo starts lying.

What the numbers mean

| Column | Meaning | | --- | --- | | Decode tok/s | Generation speed. This is the number you feel while reading output. | | Prefill tok/s | How fast it ingests your prompt. Matters enormously for long context, invisible for chat. | | Peak mem | High-water memory. The number that decides whether it runs at all. | | Spread | Fastest minus slowest across repetitions. Large spread means thermal throttling. |

Local decode is memory-bandwidth bound, not compute bound. The useful mental model is bandwidth ÷ model size ≈ ceiling tok/s, which is why a 546 GB/s laptop can beat a desktop GPU on a model too big for its VRAM, and why quantisation buys speed as well as space.

Contributing

Run the command, commit the JSON file it writes, open a PR. CI checks the schema and that your device exists in registry/devices.yml. See CONTRIBUTING.md.

Licence

Code MIT, data CC0. Benchmark numbers about publicly available hardware should not be owned by anyone.

Generated by scripts/build.mjs. Do not edit this file directly.