offgrid-ai

Privacy-first CLI for running local LLMs. Your AI, your machine, nothing leaves.

Install • Run • Done.

curl -fsSL https://raw.githubusercontent.com/eeshansrivastava89/offgrid-ai/main/install.sh | bash

What it does

You run offgrid-ai. It finds your local models, auto-configures everything, starts the server, and launches Pi. Zero configuration. No parameter tuning. No presets.

First run walks you through installing anything missing. For GGUF models, offgrid-ai installs a managed llama.cpp runtime under ~/.offgrid-ai/runtime; Homebrew is only used if you choose Homebrew-installed apps like LM Studio, Ollama, or oMLX.

offgrid-ai          # pick a model and run it
offgrid-ai status   # show running servers (from another terminal)
offgrid-ai stop     # stop a running server

Install

Recommended: one command installer

Installs Node.js if you don't have it, then installs offgrid-ai and adds it to your PATH. Prints a welcome message so you know it worked.

curl -fsSL https://raw.githubusercontent.com/eeshansrivastava89/offgrid-ai/main/install.sh | bash

Or review the install script first:

curl -fsSL https://raw.githubusercontent.com/eeshansrivastava89/offgrid-ai/main/install.sh | less

Already have Node.js?

npm install -g offgrid-ai@latest --prefer-online

This works without extra flags, but npm hides postinstall output by default, so you won't see the welcome message. Open a new terminal window or run source ~/.zshrc and then offgrid-ai.

How it works

1. Auto-detect everything. Scans for GGUF models in LM Studio and Hugging Face cache directories, and checks managed backends like Ollama/oMLX through their local APIs. Reads model metadata (quantization, context size, vision, thinking mode) directly from GGUF files. No presets, no manual configuration.

2. One command to run. offgrid-ai → pick a model → confirm context/KV memory settings on first setup → it starts llama-server, syncs Pi config, and launches Pi.

3. One model at a time. Laptops have limited RAM. One server, one model, no confusion.

Supported backends

| Backend | Type | Auto-detected | |---|---|---| | LM Studio | Visual model browser + CLI (lms) | ✓ models in ~/.lmstudio/models/ | | llama.cpp | Managed local runtime | ✓ GGUF models in ~/.lmstudio/models/ and Hugging Face cache | | llama.cpp MTP | Managed local runtime (speculative decoding) | ✓ MTP detected from model metadata | | Ollama | Managed server | ✓ via localhost:11434 | | oMLX | Managed server | ✓ via 127.0.0.1:8000 |

First run onboarding

When you run offgrid-ai for the first time on a fresh machine:

1. llama.cpp runtime — Required for GGUF models. Offered as an offgrid-ai managed runtime from official llama.cpp release binaries.
2. Pi — Required to chat from the Pi coding agent. Offered to install via npm if missing.
3. Model backend — At least one is needed (LM Studio recommended):
   • LM Studio — visual model browser + lms CLI, download models with lms get qwen/qwen3.5-9b
   • Ollama — models download on demand with ollama pull
   • oMLX — Apple Silicon optimized
4. Models — If no models found, tells you where to get them.

Homebrew is optional and only prompted when you choose a Homebrew-based backend install. Subsequent runs skip everything that's already installed. When a GGUF model is set up for the first time, offgrid-ai asks only for the memory-impacting choices: context window and KV cache precision. Sampling defaults are shown but not forced into a tuning wizard.

Data directory

~/.offgrid-ai/
  config.json          # auto-detected paths, editable for overrides
  profiles/            # one per model, auto-created on first run
    <id>/
      profile.json     # model metadata + auto-detected settings
      command.json     # llama-server flags (auto-generated, hand-editable)
      notes.md         # scratch notes
  logs/
  run/                 # PID state files
  runtime/             # managed llama.cpp binaries

Benchmark (coming soon)

"Benchmark" is always shown as an option in the CLI. If the local-llm-visual-benchmark repo is found locally, it works. If not, it offers to clone it. Model management works standalone; benchmarking is the upsell.

Development

git clone https://github.com/eeshansrivastava89/offgrid-ai.git
cd offgrid-ai
npm install
node bin/offgrid-ai.mjs

License

Personal project by Eeshan Srivastava.