@countly/ai-sdk-cohere

Countly AI observability adapter for the Cohere TypeScript SDK.

Part of the Countly AI SDK — provider-agnostic LLM observability for every AI stack.

Install

npm install @countly/ai-sdk-cohere

@countly/ai-sdk-core is pulled in automatically.

Peer dependency

cohere-ai >= 7.0.0

Quick Start (v2 API)

import { CohereClient } from "cohere-ai";
import { observeCohere } from "@countly/ai-sdk-cohere";
import { AsyncLocalStorage } from "node:async_hooks";

const userStore = new AsyncLocalStorage<{ userId: string }>();

app.use((req, res, next) => {
  userStore.run({ userId: req.user.id }, next);
});

const cohere = observeCohere(new CohereClient({ token: "..." }), {
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly-server.com",
  getDeviceId: () => userStore.getStore()?.userId,
});

const response = await cohere.v2.chat({
  model: "command-r-plus",
  messages: [{ role: "user", content: "Hello" }],
});

v1 API

const response = await cohere.chat({
  model: "command-r",
  message: "Hello",
});

Streaming

chatStream and v2.chatStream are tracked too. The row is emitted once the stream finishes, accumulated from the event stream itself (content-delta, tool-call-*, message-end for v2; stream-end for v1), so it carries the same usage, cost, tool and finish-reason data as a non-streaming call, plus latency_ttft. A stream the caller abandons half-way is recorded with status: "incomplete" — real tokens were billed, but the row only claims what was observed.

const stream = await cohere.v2.chatStream({
  model: "command-r-plus",
  messages: [{ role: "user", content: "Hello" }],
});

for await (const event of stream) {
  // ...render
}

What's captured

• Token usage — handles both v1 (meta.tokens) and v2 (usage.tokens) response shapes, plus usage_cache_read from cachedTokens. When the response reports no usage the fields are omitted (never zeroed) and the row carries usage_state: "not_reported"
• Cost (computed from model pricing; cost_priced states when a model has no price)
• Latency (latency_total, and latency_ttft for streams) in whole milliseconds
• api_host_type / api_host — which class of endpoint the client talks to (vendor_direct, gateway, local_infra, …) and its bare hostname. Path, query and credentials are never emitted
• Finish reason normalized from COMPLETE, MAX_TOKENS, STOP_SEQUENCE, TOOL_CALL, ERROR
• Tool calls (function and legacy formats) with the provider's call_id. A tool whose arguments the model emitted as malformed JSON still produces a row — the name is valid data — and the count of unparsed arguments is reported in provider_metadata
• Text output, and the model's toolPlan / thinking blocks as text_reasoning (observability level 2 only)
• Error tracking with categorization
• APM traces, per-user aggregation

Not tracked: embed, rerank, classify and the other non-generation endpoints — they are not LLM generations, so they emit no interaction row.

Caller-supplied prompt_id

By default every tracked call is stamped with an auto-generated prompt_id. If your app already owns a request/trace identifier, supply it via the getPromptId config callback and the adapter will use it verbatim (falling back to the generated id whenever the callback returns undefined):

import { AsyncLocalStorage } from "node:async_hooks";

const requestStore = new AsyncLocalStorage<{ requestId: string }>();

const cohere = observeCohere(new CohereClientV2({ token: "..." }), {
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly-server.com",
  getPromptId: () => requestStore.getStore()?.requestId,
});

The resolved id becomes the run_id of the turn — the join key carried by every row the call emits (interaction, tool, tool-parameter) — and onPrompt reports it back as prompt_id as well, so a prompt_id you already track elsewhere can be passed straight to feedback.track({ prompt_id }) without waiting for the callback. On the [CLY]_llm_interaction row itself, prompt_id now points at the row's own event_id (a generation is its own parent); use run_id to group a turn.

Feedback

User feedback (thumbs up/down, ratings, comments) is not auto-collected — wire it from your UI. Capture the prompt_id of each tracked interaction via the onPrompt callback, then record feedback against it with createFeedbackTracker (re-exported from this package, so no extra install is needed):

import { CohereClientV2 } from "cohere-ai";
import { observeCohere, createFeedbackTracker, type PromptInfo } from "@countly/ai-sdk-cohere";

const countly = { appKey: "YOUR_APP_KEY", url: "https://your-countly-server.com" };

let lastPrompt: PromptInfo | undefined;
const cohere = observeCohere(new CohereClientV2({ token: process.env.CO_API_KEY }), {
  ...countly,
  onPrompt: (info) => { lastPrompt = info; }, // fires after every tracked call
});

const feedback = createFeedbackTracker(countly, { sdk_adapter: "cohere" });

const response = await cohere.chat({
  model: "command-r-plus",
  messages: [{ role: "user", content: "Explain quantum computing" }],
});

// ...later, when the user rates the answer:
feedback.track({
  prompt_id: lastPrompt!.prompt_id,
  rating: "thumbs_up", // or "thumbs_down", or any custom string
  score: 0.9, // optional 0-1 numeric score
  category: "helpful", // optional: hallucination, irrelevant, harmful, ...
  comment: "Great answer", // optional free-form text
  deviceId: user.id, // attribute to the same user as the interaction
});

Each track() call emits a [CLY]_llm_interaction_feedback event whose prompt_id links back to the [CLY]_llm_interaction event — powering prompt → feedback funnels and per-model satisfaction breakdowns in Countly. In a real app, store prompt_id alongside the rendered message (or return it to your client) and read it back when the user rates the answer. Feedback is batched like interaction events; call feedback.flush() to send immediately, or feedback.shutdown() on process exit.

Full documentation

See the Countly AI SDK repository for the schema v2 wire contract (one row per generation, RULE A dimensions, RULE B measures with their usage_state / cost_priced markers, and the common envelope), the adapter capability matrix, observability levels (0/1/2), cost calculation, privacy controls, and Countly plugin integration (Drill, Funnels, Cohorts, APM, Crash Analytics).

License

MIT