🔁 pi-makora-provider

Open-weight models through Makora

DeepSeek V4, Gemma 4, GLM 5.2, and graduated Qwen/Kimi regression profiles for pi.

Models

| Model | ID | Reasoning | Notes | |-------|----|-----------|-------| | DeepSeek V4 Flash | deepseek-ai/DeepSeek-V4-Flash | Yes | returns reasoning field | | Gemma 4 26B A4B | google/gemma-4-26B-A4B | Yes | thinking controlled with vLLM chat_template_kwargs.enable_thinking; effort levels are not published | | GLM 5.2 FP8 | zai-org/GLM-5.2-FP8 | Yes | off via top-level reasoning_effort: "none" (only lever GLM responds to; enable_thinking/thinking/chat_template_kwargs toggles are ignored); effort high/max only (minimal/low/medium clamp to high); multi-turn continuity via clear_thinking (false when thinking on = preserve, true when off = clear; preserve_thinking is inert for GLM) injected via onPayload; returns reasoning field | | GLM 5.2 NVFP4 | zai-org/GLM-5.2-NVFP4 | Yes | off via top-level reasoning_effort: "none" (only lever GLM responds to; enable_thinking/thinking/chat_template_kwargs toggles are ignored); effort high/max only (minimal/low/medium clamp to high); multi-turn continuity via clear_thinking (false when thinking on = preserve, true when off = clear; preserve_thinking is inert for GLM) injected via onPayload; returns reasoning field | | Llama 3.3 70B FP8 | amd/Llama-3.3-70B-Instruct-FP8-KV | No | |

Installation

Option 1: Install from npm (Recommended)

pi install npm:pi-makora-provider

Then set your API key and run pi:

# Recommended: add to auth.json
# See Authentication section below

# Or set as environment variable
export MAKORA_OPTIMIZE_TOKEN=your-api-key-here

pi

Option 2: Using pi install from GitHub

Install directly from GitHub:

pi install https://github.com/monotykamary/pi-makora-provider

Then set your API key and run pi:

# Recommended: add to auth.json
# See Authentication section below

# Or set as environment variable
export MAKORA_OPTIMIZE_TOKEN=your-api-key-here

pi

Option 3: Manual Clone

1. Clone this repository:

   git clone https://github.com/monotykamary/pi-makora-provider.git
   cd pi-makora-provider

2. Set your Makora API key:

   # Recommended: add to auth.json
   # See Authentication section below
   
   # Or set as environment variable
   export MAKORA_OPTIMIZE_TOKEN=your-api-key-here

3. Run pi with the extension:

   pi -e /path/to/pi-makora-provider

Setup

API Key

Add your Makora API key to ~/.pi/agent/auth.json (recommended):

{
  "makora": { "type": "api_key", "key": "your-api-key" }
}

Or set it as an environment variable:

export MAKORA_OPTIMIZE_TOKEN=your-api-key

Usage

pi -e /path/to/pi-makora-provider

Then use /model to select from available Makora models.

Model Resolution

Models are discovered from the Makora /v1/models API and stored in models.json. Custom definitions and overrides are layered via patch.json and custom-models.json.

| File | Purpose | |---|---| | models.json | Auto-generated from Makora API (model discovery). Regenerated by node scripts/update-models.js — do not edit manually | | patch.json | Manual overrides (reasoning, compat, notes, limits, etc.) applied on top of models.json | | custom-models.json | Models not available via the API (e.g. per-slug endpoint models) |

Models are loaded by merging models.json → apply patch.json → merge custom-models.json.

Adding Custom Models

Do not edit models.json directly — it is auto-generated from the API. To customize:

• Override an existing model: Add entries to patch.json (reasoning, compat, notes, maxTokens, etc.)
• Add new models not in the API: Add entries to custom-models.json:

[
  {
    "id": "my-org/my-model",
    "name": "My Custom Model",
    "reasoning": false,
    "input": ["text"],
    "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
    "contextWindow": 131072,
    "maxTokens": 16384,
    "baseUrl": "https://inference.makora.com/my-model-slug/v1"
  }
]

Preserved Thinking

Reasoning models on Makora keep their chain-of-thought across turns by default: each turn's reasoning trace is rendered into the next prompt so the model recalls its prior deduction (better multi-turn recall for coding). By default this follows the thinking switch — preserve is on when thinking is on, and cleared when thinking is off — via the { "$var": "thinking.enabled" } schema in patch.json.

Use /makora-settings to pin preserved thinking on or off per model, independently of the thinking switch:

• Preserve Thinking — keep every turn's reasoning trace in the next prompt. Suited for coding, but can overthink on prose and costs tokens.
• Clear Thinking — let the template drop older reasoning each turn. Lighter and often better for prose, but can hurt multi-turn recall.

Selections persist to ~/.pi/agent/extensions/makora.json (modelOverrides, deep-merged on top of patch.json) and take effect immediately — the provider is re-registered on toggle, and a notification on model switch reports the current state.

| Model | Flag | |---|---| | GLM 5.2 FP8 / NVFP4 | clear_thinking (false = preserve) | | Gemma 4 26B A4B | enable_thinking follows the thinking switch | | Qwen 3.6 27B / 35B A3B (deprecated grace layer) | preserve_thinking | | Kimi K2.7 Code (deprecated grace layer) | namespaced thinking + preserve_thinking |

You can also hand-edit makora.json directly:

{
  "modelOverrides": {
    "zai-org/GLM-5.2-FP8": { "compat": { "chatTemplateKwargs": { "clear_thinking": true } } },
    "google/gemma-4-26B-A4B": { "compat": { "chatTemplateKwargs": { "enable_thinking": false } } },
    "unsloth/Qwen3.6-27B-NVFP4": { "compat": { "chatTemplateKwargs": { "preserve_thinking": false } } }
  }
}

API Notes

• Each model is accessible at https://inference.makora.com/v1/chat/completions (unified endpoint)
• Models with a baseUrl override use their per-slug endpoint instead
• The API is OpenAI-compatible (chat completions format)
• All models are hosted on vLLM
• The developer role is not supported (prompts are silently dropped); supportsDeveloperRole is set to false for all models

NaN-Collapse Guard

The GLM-5.2 NVFP4/FP8 quants have an engine-side bug: at long context (from ~9k prompt tokens) the vLLM NVFP4 MoE prefill produces NaN logits, so the reasoning trace collapses into a single token repeated indefinitely (!!!!… here, {},{},{},… on other deployments), finish_reason: length. Requesting logprobs surfaces the root cause as HTTP 400 Out of range float values are not JSON compliant: nan. This is a vLLM numerical bug (matches vLLM #31856 / #47042), not a model or prompt issue.

This extension ships a guard (nan-collapse-guard.ts) scoped to those quants. It detects the collapse as a NaN-argmax onset fixed point — the first ~64 chars of the reasoning trace being one short unit (!, {},, ();, …) repeated — instead of the old blanket repetition pattern-matcher, and recovers by trimming the degenerate turn and queueing a hidden AgentSession follow-up with backoff. The marker is removed before provider serialization, and recovery retries until a clean turn, user abort, or session change. It does not trim valid context — only the degenerate turn is removed, so the user's session is preserved. The recurrence is the engine bug re-triggering as context grows; the guard catches each collapse early (at onset, ~4 tokens) instead of after a 15k-token ! run.

Set MAKORA_NAN_CANARY=1 to enable an optional preflight canary that probes the payload with logprobs:true and logs a nan flag for the engine team (off by default; adds latency on long contexts).