llmizeOFF

Formerly offllama. Now packaged as llmizeoff — a smarter, more production-ready self-hosted LLM runtime and toolkit. Migrating? npm install llmizeoff and change imports from offllama → llmizeoff. The old OffLlamaClient / OffLlamaError names still work as aliases.

Try it live → zulqurnainj.com/chat · Nayab is the hosted demo of llmizeOFF

Run LLM inference on any host — cPanel shared hosting, VPS, Raspberry Pi, Android — with zero cloud dependencies, no subscriptions, and no external lock-in.

Self-hosted · Offline-first · VPS & cPanel ready · Android compatible · No subscriptions

What is llmizeOFF?

llmizeOFF (npm: llmizeoff) is an open-source LLM runtime designed to run where cloud AI cannot:

• $5/month VPS — works on the smallest DigitalOcean or Hetzner droplets
• cPanel shared hosting — deploys as a Node.js app without root access
• Android apps — native JNI/NDK module for fully offline on-device inference
• Local machines — zero-config CLI for developer tools and scripts
• Web apps — universal HTTP client for browser, React Native, and Node.js

No GPU required. No monthly API bills. No data sent to third parties.

Live demo — Nayab

Nayab is the hosted demo of llmizeOFF. It runs Qwen 2.5-1.5B on a standard VPS with real token streaming. Try it free to see what llmizeOFF can do in a production environment.

What's inside

| Export | Environment | Description | |--------|-------------|-------------| | llmizeoff (default) | Node.js / cPanel | Embedded inference via node-llama-cpp + OpenAI-compatible HTTP server | | llmizeoff/client | Browser, RN, Node.js, Deno, Bun | Zero-dependency HTTP client for any llmizeOFF server | | llmizeoff/react-native | React Native (iOS + Android) | Offline on-device inference via llama.rn OR HTTP fallback | | llmizeoff/nano | Any | Zero-model regex extraction + template message builders | | android/ | Kotlin / Android native | Full JNI/NDK library — 100% offline, no server needed |

Quick start (Node.js / VPS / cPanel)

npm install llmizeoff

# Download recommended model (Qwen 2.5-1.5B ~1.1GB — best quality/speed on CPU)
npx llmizeoff download

# Start OpenAI-compatible server with real token streaming
npx llmizeoff serve --port 8080 --api-key my-secret

Call it from anywhere:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-secret" \
  -d '{
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

Recommended models for self-hosting

Tested on a 6-core AMD EPYC VPS (12GB RAM, no GPU):

| Model | Size | First token | Tokens/sec | Verdict | |-------|------|-------------|------------|---------| | Qwen 2.5-1.5B Q4_K_M | 1.1 GB | 3-5s | 4-6 | Best for VPS | | Qwen 2.5-0.5B Q4_K_M | 469 MB | 2-3s | 8-12 | Fastest, limited quality | | Qwen 2.5-3B Q4_K_M | 2.0 GB | 6-10s | 2-3 | Better quality, slower | | Phi-3.5-mini 3.8B | 2.3 GB | 15-20s | 1-2 | Too slow for interactive chat |

Qwen 2.5-1.5B is the sweet spot: smart enough for real tasks, fast enough for streaming chat.

For sub-1-second responses, pair llmizeOFF with Groq (free tier, 800+ tok/s).

Embed in Next.js / Express

import { LlamaEngine } from "llmizeoff";

const llama = new LlamaEngine({ contextSize: 2048 });
await llama.load(); // auto-downloads Qwen 2.5-1.5B on first run

const reply = await llama.chat([
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "What is the capital of France?" }
]);

console.log(reply); // "The capital of France is Paris."

Android — fully offline on-device

// build.gradle
dependencies {
    implementation("com.github.Zulqurnain:llmizeoff-android:0.3.1")
}

val engine = LlmizeOffEngine(context, modelPath = "models/qwen2.5-0.5b-q4_k_m.gguf")
engine.load()
val reply = engine.chat("Explain recursion in one sentence.")

Universal HTTP client

Works in browser, React Native, Node.js, Deno, and Bun — zero dependencies:

import { LlmizeOffClient } from "llmizeoff/client";

const client = new LlmizeOffClient({
  baseUrl: "https://your-server.example.com:8080",
  apiKey: "my-secret",
});

// Streaming
for await (const token of client.streamChat([
  { role: "user", content: "Write a haiku about self-hosting" }
])) {
  process.stdout.write(token);
}

VPS deployment (PM2 + nginx)

# 1. Install
npm install llmizeoff pm2 -g

# 2. Download model
npx llmizeoff download --model qwen2.5-1.5b

# 3. Start with PM2
pm2 start node_modules/llmizeoff/dist/server.js \
  --name llmizeoff \
  --env PORT=8080,HOSTNAME=127.0.0.1,OFFLLAMA_API_KEY=your-key

pm2 save && pm2 startup

Add nginx reverse proxy to expose on your domain. See the Nayab source for a complete production example.

Why llmizeOFF?

| Feature | llmizeOFF | Cloud AI APIs | |---------|-----------|---------------| | Monthly cost | $0 (your VPS) | $20-100+/month | | Data privacy | 100% local | Sent to cloud | | Works offline | Yes | No | | VPS / cPanel | Yes | No | | Android (offline) | Yes | No | | Vendor lock-in | None | Yes |

Roadmap — llmizeOFF Pro (coming soon)

The core runtime stays open-source and free forever. The upcoming Pro edition adds:

• Visual dashboard for model management
• One-click model download and switching
• Multi-user support with rate limiting
• Android SDK (AAR package)
• Priority support and SLAs

Support development and shape the roadmap:

☕ Ko-fi: @zulqurnainjj

License

MIT © Zulqurnain Haider