@hyperlite-ai/hyperlite-pi

Offline AI chat for the Raspberry Pi 5. Native ARM64 binary, zero cloud, runs in your terminal.

What is this

HyperLite-PI is a terminal-based AI chat client built in Rust, optimised specifically for the Raspberry Pi 5 and ARM64 single-board computers. It runs large language models entirely on-device — no API keys, no internet connection required after setup, no data leaving your machine.

Requirements

• Raspberry Pi 5 (or any ARM64 Linux SBC)
• Debian Trixie / Ubuntu 24.04 or later (aarch64)
• Node.js ≥ 16
• ~2 GB free disk space for a small model

Install

npm install -g @hyperlite-ai/hyperlite-pi
hl

On first launch HyperLite-PI will:

1. Detect your hardware (RAM, CPU cores)
2. Show recommended models for your Pi
3. Download your chosen model from HuggingFace
4. Compile a native ARM64 inference server from llama.cpp source (~15 min, once only)

After that, launching hl takes under 2 seconds.

Why native ARM64

Most pre-built inference binaries for Linux ARM64 are Cosmopolitan APE format — x86_64 code that runs under QEMU emulation on your Pi. HyperLite-PI compiles llama-server directly on your hardware with GGML_NATIVE=ON, enabling the Cortex-A76 NEON and DotProd instruction sets. The difference is 5–10× faster token generation compared to emulated binaries.

Performance (Pi5, 16 GB, native binary)

| Model | Size | Speed | |---|---|---| | SmolLM2 1.7B Q4_K_M | ~1 GB | 35–50 tok/s | | Qwen2.5 3B Q4_K_M | ~2 GB | 22–32 tok/s | | Llama 3.2 3B Q4_K_M | ~2 GB | 20–30 tok/s | | Mistral 7B Q4_K_M | ~4 GB | 10–14 tok/s | | Llama 3.1 8B Q4_K_M | ~5 GB | 9–13 tok/s |

Features

• Fully offline — models and runtime live on your SSD, nothing phoned home
• Streaming responses — tokens appear as they are generated
• Markdown rendering — code blocks with syntax highlighting in the terminal
• Tool use — the model can read files, run shell commands, search the web (DuckDuckGo, no key needed), and edit code
• Session history — all conversations saved locally in SQLite
• Multiple agents — switch between General, Coding, and Plan modes or create your own
• Model picker — browse and download models from inside the app (Ctrl+M)
• Reasoning model support — thinking tokens from DeepSeek-R1, QwQ, etc.

Keybindings

| Key | Action | |---|---| | Enter | Send message | | Alt+Enter | New line | | Ctrl+M | Model picker | | Ctrl+S | Session list | | Ctrl+N | New session | | Ctrl+K | Command palette | | Ctrl+A | Agent picker | | Ctrl+C | Stop generation | | Ctrl+Q | Quit | | ? | Help |

Configuration

# ~/.config/hyperlite/settings.toml
theme   = "dracula"
sidebar = "auto"

[[permissions.rules]]
tool    = "bash"
pattern = "git *"
action  = "allow"

Links

• hyperlite.org

License

MIT