@bunnyosai/worker

Native inference worker for bunnyOS. Runs LLMs locally via Ollama and serves streaming inference jobs for the bunnyOS network.

Requirements

• Node.js 18.17+ (22+ recommended)
• Ollama installed — the worker starts it and pulls the model automatically on first run
• ~6 GB disk for the model (gemma4:e4b-it-qat)
• A GPU / Apple Silicon

Get a key

The worker authenticates with a key. Get one from the bunnyOS dashboard:

1. Go to platform.bunnyos.ai
2. Open Settings → Workers
3. Create a worker and copy its key — it's shown once, so save it.

That key is all the worker needs to run.

Usage

npx @bunnyosai/worker@latest --key <key>

Or via the environment / a saved creds file:

export BUNNYOS_WORKER_KEY=…
npx @bunnyosai/worker@latest                           # picks the key up from the env
npx @bunnyosai/worker@latest --save-creds --key <key>  # persist to ~/.bunnyos/worker.json

That's it — it starts Ollama if needed, pulls the model, then loads and warms it up: before connecting it runs a short generation to warm the GPU and measure decode speed, retrying while the machine is still cold. This keeps the broker's admission probe from being your first (cold, possibly-too-slow) generation. Then it connects and begins serving. Press Ctrl-C to stop.

Options

| flag | env | default | meaning | |------|-----|---------|---------| | --key | BUNNYOS_WORKER_KEY | — | worker key (required) | | --model | BUNNYOS_MODEL | gemma-4-e4b | model to serve | | --api | BUNNYOS_API | https://api.bunnyos.ai | broker base URL | | --ollama-url | OLLAMA_HOST | http://localhost:11434 | Ollama base URL | | --no-pull | — | off | don't auto-pull the model if missing | | --save-creds | — | off | save the key to ~/.bunnyos/worker.json | | --health-port | BUNNYOS_HEALTH_PORT | off | serve /healthz + /status on this port | | --health-host | BUNNYOS_HEALTH_HOST | 127.0.0.1 | health bind host (0.0.0.0 for k8s) | | --worker-id | BUNNYOS_WORKER_ID | — | optional; display only | | --label | — | — | informational label | | — | BUNNYOS_MIN_TPS | 8 | warmup decode floor (tok/s) required before connecting |

Set BUNNYOS_LOG_LEVEL=debug for verbose logging.

Images

When the model build reports the vision capability, the worker loads the multimodal portion at startup and serves image content parts on user/system messages. Images must be inline — a data:image/...;base64,... URL or a bare base64 string. Remote http(s) image URLs are dropped, never fetched: the worker would be dereferencing consumer-supplied URLs from inside your network.

Health checks

Pass --health-port <n> to expose two HTTP endpoints — useful when running the worker under systemd, Docker, or Kubernetes. Bound to 127.0.0.1 by default; use --health-host 0.0.0.0 for k8s probes.

• GET /healthz — liveness. Returns 200 as long as the process is responsive; it stays 200 even during a reconnect. Point a restart policy at this — a failure means the process is wedged.

• GET /status — readiness + observability. Returns 200 when the worker is connected, admitted, Ollama is reachable, and the broker is still actively talking to us (brokerFresh); 503 otherwise. brokerFresh catches a silent eviction — where the broker drops the worker from its pool without closing the socket — so a dropped worker reports 503 instead of a false 200. Wire this to monitoring/dashboards — do not restart on 503; the worker detects the drop and reconnects on its own. Body:

  {
    "state": "admitted",
    "connected": true,
    "admitted": true,
    "brokerFresh": true,
    "ollamaOk": true,
    "model": "gemma-4-e4b",
    "uptimeSec": 1234,
    "jobsServed": 42,
    "totalOutputTokens": 9001,
    "lastJobAt": "2026-07-06T12:00:00.000Z"
  }

Troubleshooting

• "Ollama is not installed" — install it from https://ollama.com/download and re-run.
• Evicted too_slow — your machine is decoding below the network's speed floor. The worker warms up and measures decode speed before connecting and logs it (warmup N/5: … tok/s, 100% GPU); if it also warns the model is on CPU or a partial GPU split, that's the cause. Close other GPU-heavy apps, or use a machine with more VRAM / a faster GPU.
• Slow / stuck on a shared cloud GPU — if warmup logs a low tok/s or a CPU/partial placement, the model didn't fully fit on the GPU. Watch the ollama: … lines at startup for no compatible GPUs/CUDA errors.
• Frequent reconnects — usually a flaky network between you and the broker; the worker reconnects automatically and keeps serving.