@holokai/holo-provider-cohere
v0.1.1
Published
Cohere rerank provider plugin for Holo LLM Gateway
Readme
@holokai/holo-provider-cohere
Cohere rerank provider plugin for the Holo LLM Gateway. Rerank-only family: one provider can be associated alongside an OpenAI-family plugin (multi-plugin providers) so a single vLLM endpoint serves chat and Cohere-compatible rerank from one baseUrl, or a hosted Cohere provider can serve rerank standalone.
Protocols
| Protocol | Capability | Wire paths |
|-----------------|------------|-------------------------------------|
| cohere.rerank | rerank | /rerank, /v1/rerank, /v2/rerank |
| cohere.models | models | /v1/models |
Rerank is non-streaming and native-only: cross-family translation is
deliberately unsupported (see docs/decisions/ADR-HOLO-007 in the monorepo).
Connection config
| Field | Default | Notes |
|--------------|--------------------------|-----------------------------------------|
| baseUrl | https://api.cohere.com | Any Cohere-compatible server (vLLM) |
| apiKey | — | Encrypted at rest; omit for local vLLM |
| rerankPath | /v2/rerank | Use /v1/rerank for older servers |
Billing
Cohere bills rerank per search unit, not per token. The auditor records
search_units from meta.billed_units in the response metrics, and the
pricing dataset prices them via token_costs.search_unit.
Test
pnpm --filter @holokai/holo-provider-cohere test