pi-nvidia-nim-diegovisk

NVIDIA NIM provider for pi with the full chat model catalog, per-family thinking/request transforms, and aggressive rate-limit retry for uninterrupted free-tier development.

Forked from pi-extension-nvidia-nim (MIT, Tibbee) with merged pi-nvidia-rate-limit-retry (MIT, Diegovisk).

Why this fork

Pi's built-in nvidia provider ships ~20 models with no per-model thinking transforms. The upstream pi-extension-nvidia-nim package registers nvidia-nim with ~82 chat models and 8+ thinking handlers, but does not retry 429s aggressively. This fork combines both.

Recommended free-dev stack

| Role | Model | |------|-------| | Auto failover (sticky router) | nvidia-nim/nvidia-router | | Primary (tools + vision) | nvidia-nim/minimaxai/minimax-m3 | | Reasoning fallback | nvidia-nim/nvidia/nemotron-3-super-120b-a12b | | Fast fallback | nvidia-nim/meta/llama-3.3-70b-instruct |

Set defaultModel to nvidia-router to let the extension pick from the chain above and only switch on rate-limit exhaustion or context overflow.

Install

npm (recommended)

pi install npm:@diegovisk/[email protected]

Or add to ~/.pi/agent/settings.json:

{
  "packages": ["npm:@diegovisk/pi-nvidia-nim"]
}

Note: Published as @diegovisk/pi-nvidia-nim because unscoped pi-nvidia-nim is taken on npm.

git

pi install git:github.com/Diegovisk/pi-nvidia-nim@main

Local path (development)

pi install file:C:/Users/diego/projects/pi-nvidia-nim

Configure API key

Set one of:

export NVIDIA_NIM_API_KEY="nvapi-..."
# or (compatible with pi /login and auth.json "nvidia" entry)
export NVIDIA_API_KEY="nvapi-..."

Switch default provider

In ~/.pi/agent/settings.json:

{
  "defaultProvider": "nvidia-nim",
  "defaultModel": "nvidia-router",
  "packages": ["file:C:/Users/diego/projects/pi-nvidia-nim"]
}

Or pin a concrete model:

{
  "defaultProvider": "nvidia-nim",
  "defaultModel": "minimaxai/minimax-m3"
}

Remove npm:pi-extension-nvidia-nim and the standalone ~/.pi/agent/extensions/nvidia-rate-limit-retry.ts to avoid double-wrapping.

Then /reload in pi.

Features

• ~82 curated chat models from scraped NIM metadata + 44 regex families
• Per-model request transforms via before_provider_request (DeepSeek V4, GLM, MiniMax M3, Nemotron, Kimi K2.6, Seed OSS, etc.)
• Content normalization for legacy models that reject structured content arrays
• Rate-limit retry — up to 20 attempts with exponential backoff (env-tunable); covers nvidia, nvidia-nim, and OpenRouter nvidia/*
• Compaction-safe summarization — auto-compact and /compact calls force thinking off so MiniMax/NIM does not 400 during context overflow recovery
• NVIDIA Router — virtual nvidia-router model with sticky per-session routing across the free-dev fallback chain; re-selects only on 429 exhaustion or context overflow
• Catalog refresh — /nim-refresh caches live API model list; npm run refresh-metadata merges new IDs into metadata

Commands

| Command | Description | |---------|-------------| | /nim-models | List registered models by family + recommended stack | | /nim-refresh | Refresh ~/.pi/nvidia-nim-cache.json from NIM API | | /nim-router | Show sticky routed model, fallback chain, and eligibility | | /nim-router reset | Clear sticky router state (re-pick from chain head) | | /nim-retry-status | Show rate-limit retry stats for this session + audit log path | | /nim-retry-status reset | Clear retry session counters |

Retry events are appended to ~/.pi/nvidia-nim-retry.log (JSONL). Disable with NVIDIA_RETRY_AUDIT=0.

Router env vars

| Variable | Default | Meaning | |----------|---------|---------| | NVIDIA_ROUTER_CHAIN | minimaxai/minimax-m3,nvidia/nemotron-3-super-120b-a12b,meta/llama-3.3-70b-instruct | Comma-separated fallback order | | NVIDIA_ROUTER_MARGIN | 1.15 | Context headroom multiplier for eligibility filter |

Router behavior:

• Picks one concrete model per session (sticky) from the chain.
• Only advances on 429 retry budget exhausted or context overflow.
• Filters candidates by estimated context size and vision requirements.
• Advertises 1M context on the virtual model so pi does not pre-reject long sessions.

Verify after /reload:

[nvidia-router] loaded — sticky failover router for nvidia-router.

Rate-limit env vars

| Variable | Default | Meaning | |----------|---------|---------| | NVIDIA_RETRY_MAX | 20 | Max attempts per request | | NVIDIA_RETRY_BASE_MS | 2000 | Base backoff | | NVIDIA_RETRY_CAP_MS | 60000 | Max single delay | | NVIDIA_RETRY_JITTER | 0.2 | Jitter fraction | | NVIDIA_RETRY_AUDIT | 1 | Set 0 to disable JSONL audit file | | NVIDIA_RETRY_AUDIT_LOG | ~/.pi/nvidia-nim-retry.log | Audit log path |

Verify after /reload:

[nvidia-rate-limit-retry] loaded — NVIDIA NIM retries: max 20, base 2000ms, cap 60000ms, jitter 20%.

Debug

NIM_DEBUG=1 pi

Inspect ~/.pi/nim-debug.log for final request payloads per model.

Validation matrix

Smoke-test these families at thinking levels off, medium, high:

| Family | Example model | Pass criteria | |--------|---------------|---------------| | minimax-inline | minimaxai/minimax-m3 | Tool calls; thinking toggles | | nemotron-3-super-effort | nvidia/nemotron-3-super-120b-a12b | Reasoning streams; no 400 | | deepseek-v4 | deepseek-ai/deepseek-v4-flash | chat_template_kwargs in debug log | | qwen-chat-template | qwen/qwen3.5-122b-a10b | Native pi compat sufficient | | glm | z-ai/glm-5.2 | enable_thinking / clear_thinking correct | | Plain | meta/llama-3.3-70b-instruct | No structured-array 400 | | Rate limit | any free model | [nvidia-rate-limit-retry] logs; succeeds by attempt 2–3 |

Run unit tests:

npm test

Upstream

• Tibbee/pi-nvidia-nim-provider — base provider + handlers
• Diegovisk/pi-nvidia-rate-limit-retry — deprecated; retry is bundled in this repo

License

MIT — see LICENSE.