Provider Retry Proxy

An OMP/Pi extension that runs a local HTTP retry proxy for LLM provider requests, with configurable per-provider triangular backoff and infinite retry support.

Designed for scenarios where model provider resources are scarce and frequently return 503/429/529 errors — the proxy aggressively retries with a short, capped delay to seize the provider's available capacity window.

How it works

OMP agent ──→ 127.0.0.1:7878/<provider>/<path> ──→ Retry Proxy ──→ upstream provider API
                    │
                    ├─ Transparent passthrough (SSE streaming, headers, body)
                    ├─ Triangular backoff retry on 429/502/503/504/529
                    ├─ Network error retry (ETIMEDOUT, ECONNRESET, terminated, ...)
                    ├─ Permanent error exclusion (daily/monthly quota, billing, context-length)
                    └─ Infinite retry mode (maxAttempts: -1, 30-min elapsed guard)

The proxy runs in-process within one OMP session, shared by all concurrent sessions:

• Multiple concurrent OMP sessions share the same proxy instance (first session to start owns it).
• When the owning session exits, other sessions detect the proxy is gone via turn_start and restart it in-process — no timing gap.
• A 2s background health probe also catches proxy death while a session is idle.
• No external runtime dependency; the proxy lives inside the OMP process.

Triangular backoff

The delay formula (ported from GCMP, MIT):

delay = min(initialDelayMs * triangular(n), maxDelayMs)
where triangular(n) = n * (n + 1) / 2

With initialDelayMs: 20, maxDelayMs: 50:

| Attempt | Calculation | Delay | |---------|-------------|-------| | 1 | 20 * 1 | 20ms | | 2 | 20 * 3 = 60 -> cap | 50ms | | 3 | 20 * 6 = 120 -> cap | 50ms | | 4+ | -> cap | 50ms |

From the 2nd retry onward, the delay is a constant 50ms — fast enough to seize a capacity window without hammering.

Error classification

The proxy retries the following error types:

| Category | Signals | |----------|---------| | Rate limit | HTTP 429/529, rate_limit_exceeded, too_many_requests, quota_exceeded, resource_exhausted, throttled | | Server error | HTTP 502/503/504, EngineInternalError, "service unavailable", "system is busy" | | Network error | terminated, ETIMEDOUT, ECONNRESET, ECONNREFUSED, fetch failed, socket hang up, EOF |

Permanent errors are excluded even if they carry a 429 status code:

• Daily/monthly quota exhaustion ("per day", "daily quota")
• Billing/plan issues ("billing", "upgrade your plan")
• Context length exceeded ("context length", "prompt too long")

Installation

Via omp plugin link (local development)

cd /path/to/provider_retry_proxy
bun build src/index.ts --outdir dist --target bun
bun build src/proxy.ts --outdir dist --target bun
omp plugin link .

Via npm (when published)

pi install npm:provider-retry-proxy

Configuration

1. Proxy config — ~/.omp/agent/retry-proxy.json

Copy retry-proxy.example.jsonc to ~/.omp/agent/retry-proxy.json and fill in your provider's upstream URL:

cp retry-proxy.example.jsonc ~/.omp/agent/retry-proxy.json

2. OMP models config — ~/.omp/agent/models.yml

Change the provider's baseUrl to point at the proxy, prefixing the path with the provider key:

providers:
  xfyun-anthropic:
    baseUrl: http://127.0.0.1:7878/xfyun   # was: https://your-provider-host/path
    # ... all other fields unchanged

3. OMP retry config — ~/.omp/agent/config.yml

Keep OMP's built-in retry enabled as a fallback for mid-stream disconnects (the proxy can only retry before streaming begins):

retry:
  enabled: true       # Fallback for stream-mid-disconnect errors
  maxRetries: 3       # Proxy handles most 503s; OMP only catches stream breaks
  baseDelayMs: 1000   # Stream-break retry needs full re-send, slower is fine
  maxDelayMs: 0       # Disable OMP's hard-fail cap (0 = no cap)

How retry layers interact

Request flow:
  OMP -> Proxy (fast retry: 50ms, handles 503/429/529/network)
           | (2xx Response obtained)
        Upstream (SSE streaming begins)
           | (if stream breaks mid-way)
        OMP retry (slow fallback: 1s, re-sends full conversation)

| Error scenario | Handled by | Why | |----------------|-----------|-----| | Provider returns 503/429/529 | Proxy (50ms) | Fast retry before OMP even sees an error | | Network timeout / connection refused | Proxy (50ms) | Same | | 200 OK, then SSE stream breaks | OMP retry (1s) | Proxy cannot retry mid-stream without duplicating tokens | | Daily/monthly quota exhausted | Neither | Permanent error, classified and excluded from retry |

Standalone usage

The proxy can run independently without OMP:

RETRY_PROXY_CONFIG=~/.omp/agent/retry-proxy.json bun run dist/proxy.js

Acknowledgements

The retry classification logic (retryClassifier.ts) and triangular backoff retry manager (retryManager.ts) are ported from GCMP by VicBilibily, licensed under the MIT License. See the file headers and LICENSE file for the full MIT notice.

License

MIT