@dtmirizzi/pi-openrouter-multimodal

OpenRouter multimodal tool integration for Pi. Provides 8 independently toggleable tools with per-modality model selection and session-persistent settings.

| Tool | What it does | |------|-------------| | web_search | Server-side web search with real-time results | | web_fetch | Fetch page content from a URL (web, docs, PDFs) | | image_generate | Text-to-image generation via OpenRouter chat completions | | image_understand | Analyze images via vision models | | video_understand | Analyze videos (YouTube links work with Gemini) | | pdf_read | Extract and analyze PDF content | | tts_speak | Text-to-speech via OpenRouter /audio/speech endpoint | | stt_transcribe | Speech-to-text via OpenRouter /audio/transcriptions endpoint |

Install

pi install npm:@dtmirizzi/pi-openrouter-multimodal

Or from a local checkout:

pi install /path/to/pi-openrouter-multimodal

API Key

The extension resolves the OpenRouter API key from (priority order):

1. OPENROUTER_API_KEY environment variable
2. Pi model registry (provider openrouter)
3. ~/.pi/agent/models.json under providers.openrouter.apiKey

Commands

| Command | Description | |---------|-------------| | /web-tools | Toggle tools on/off and configure search/fetch engines | | /web-models | Select models per modality (image, vision, video, PDF, TTS voice, STT) | | /web-search | Toggle web_search and configure search engine | | /web-fetch | Toggle web_fetch and configure fetch engine |

Each command opens an interactive overlay. Use ↑↓ to navigate, ←→ to cycle values, Esc to close. Settings persist across sessions and survive compaction, shutdown, and tree navigation.

/web-tools

Toggle each tool on/off and set search/fetch engine preferences. Also includes a verbose/compact status-bar display toggle.

/web-models

Select the model for each modality from a list fetched live from the OpenRouter API at startup. Falls back to a comprehensive built-in list if the API is unavailable.

Tools

web_search

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | query | string | required | Search query | | engine | string | auto | auto, native, exa, firecrawl, parallel | | max_results | integer | 5 | Results per search (1-25) | | search_context_size | string | — | low (5K), medium (15K), high (30K) | | allowed_domains | string[] | — | Only return results from these domains | | excluded_domains | string[] | — | Exclude results from these domains |

web_fetch

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | url | string | required | URL to fetch content from | | engine | string | auto | auto, native, exa, openrouter, firecrawl, parallel | | max_content_tokens | integer | — | Max content length (approximate tokens) |

image_generate

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | prompt | string | required | Text prompt describing the image | | model | string | state | Override the default model from /web-models |

Selected via /web-models. Models are fetched live from OpenRouter at startup; fallback list includes Gemini Flash Image, GPT-5 Image, FLUX.2, Seedream, Riverflow, Recraft, Grok Imagine, and more.

image_understand

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | url | string | required | Image URL or base64 data URL | | prompt | string | Describe this image in detail | Analysis prompt | | model | string | state | Override default from /web-models |

video_understand

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | url | string | required | Video URL (YouTube links work with Gemini) | | prompt | string | Describe what happens in this video | Analysis prompt | | model | string | state | Override default from /web-models |

pdf_read

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | url | string | required | URL of the PDF document | | prompt | string | Summarize this document | Analysis prompt | | model | string | state | Override default from /web-models | | engine | string | cloudflare-ai | cloudflare-ai (free), mistral-ocr (scanned docs), or native |

tts_speak

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | text | string | required | Text to convert to speech | | model | string | state | Override default from /web-models | | voice | string | state | Override default from /web-models |

stt_transcribe

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | audio | string | required | Base64-encoded audio data | | format | string | required | wav, mp3, flac, m4a, ogg, webm, aac | | model | string | state | Override default from /web-models | | language | string | — | ISO-639-1 language code (optional) |

How It Works

All tools proxy requests through OpenRouter's API:

• web_search / web_fetch — Chat completions with server tool definitions (openrouter:web_search / openrouter:web_fetch)
• image_generate — Chat completions with modalities: ["image", "text"] on the selected image generation model
• image_understand / video_understand — Chat completions with multimodal content blocks (image_url, video_url)
• pdf_read — Chat completions with file content block and file-parser plugin
• tts_speak — Direct call to /api/v1/audio/speech
• stt_transcribe — Direct call to /api/v1/audio/transcriptions

Model Discovery

On startup, the extension fetches available models from GET /api/v1/models?output_modalities=... and caches them for use in the /web-models settings panel. If the API is unreachable, a comprehensive set of fallback models is used.

Development

# Install dependencies
npm install

# Run tests
npm test               # all tests
npm run test:unit      # unit tests only
npm run test:integration  # requires OPENROUTER_API_KEY

# Format and lint
npm run fmt            # format all files
npm run lint           # lint + fix all files
npm run check          # format + lint + organize imports
npm run check:ci       # strict CI check (format + lint, no writes)

The repo uses Biome for formatting and linting. CI enforces both on every push and PR.

Assets

References

• OpenRouter Web Search docs
• OpenRouter Web Fetch docs
• OpenRouter Multimodal overview
• OpenRouter Image Generation
• OpenRouter TTS
• OpenRouter List Models API