@countly/ai-sdk-cohere

Countly AI observability adapter for the Cohere TypeScript SDK.

Part of the Countly AI SDK — provider-agnostic LLM observability for every AI stack.

Install

npm install @countly/ai-sdk-cohere

@countly/ai-sdk-core is pulled in automatically.

Peer dependency

cohere-ai >= 7.0.0

Quick Start (v2 API)

import { CohereClient } from "cohere-ai";
import { observeCohere } from "@countly/ai-sdk-cohere";
import { AsyncLocalStorage } from "node:async_hooks";

const userStore = new AsyncLocalStorage<{ userId: string }>();

app.use((req, res, next) => {
  userStore.run({ userId: req.user.id }, next);
});

const cohere = observeCohere(new CohereClient({ token: "..." }), {
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly-server.com",
  getDeviceId: () => userStore.getStore()?.userId,
});

const response = await cohere.v2.chat({
  model: "command-r-plus",
  messages: [{ role: "user", content: "Hello" }],
});

v1 API

const response = await cohere.chat({
  model: "command-r",
  message: "Hello",
});

What's captured

• Token usage (input_tokens, output_tokens) — handles both v1 (meta.tokens) and v2 (usage.tokens) response shapes
• Cost (computed from model pricing)
• Latency
• Finish reason normalized from COMPLETE, MAX_TOKENS, STOP_SEQUENCE, TOOL_CALL, ERROR
• Tool call extraction (function and legacy formats)
• Error tracking with categorization
• APM traces, per-user aggregation

Caller-supplied prompt_id

By default every tracked call is stamped with an auto-generated prompt_id. If your app already owns a request/trace identifier, supply it via the getPromptId config callback and the adapter will use it verbatim (falling back to the generated id whenever the callback returns undefined):

import { AsyncLocalStorage } from "node:async_hooks";

const requestStore = new AsyncLocalStorage<{ requestId: string }>();

const cohere = observeCohere(new CohereClientV2({ token: "..." }), {
  appKey: "YOUR_APP_KEY",
  url: "https://your-countly-server.com",
  getPromptId: () => requestStore.getStore()?.requestId,
});

The resolved prompt_id is the same value stamped on the [CLY]_llm_interaction event, so a prompt_id you already track elsewhere becomes the join key for feedback correlation — pass it straight to feedback.track({ prompt_id }) without waiting for the onPrompt callback.

Feedback

User feedback (thumbs up/down, ratings, comments) is not auto-collected — wire it from your UI. Capture the prompt_id of each tracked interaction via the onPrompt callback, then record feedback against it with createFeedbackTracker (re-exported from this package, so no extra install is needed):

import { CohereClientV2 } from "cohere-ai";
import { observeCohere, createFeedbackTracker, type PromptInfo } from "@countly/ai-sdk-cohere";

const countly = { appKey: "YOUR_APP_KEY", url: "https://your-countly-server.com" };

let lastPrompt: PromptInfo | undefined;
const cohere = observeCohere(new CohereClientV2({ token: process.env.CO_API_KEY }), {
  ...countly,
  onPrompt: (info) => { lastPrompt = info; }, // fires after every tracked call
});

const feedback = createFeedbackTracker(countly, { sdk_adapter: "cohere" });

const response = await cohere.chat({
  model: "command-r-plus",
  messages: [{ role: "user", content: "Explain quantum computing" }],
});

// ...later, when the user rates the answer:
feedback.track({
  prompt_id: lastPrompt!.prompt_id,
  rating: "thumbs_up", // or "thumbs_down", or any custom string
  score: 0.9, // optional 0-1 numeric score
  category: "helpful", // optional: hallucination, irrelevant, harmful, ...
  comment: "Great answer", // optional free-form text
  deviceId: user.id, // attribute to the same user as the interaction
});

Each track() call emits a [CLY]_llm_interaction_feedback event whose prompt_id links back to the [CLY]_llm_interaction event — powering prompt → feedback funnels and per-model satisfaction breakdowns in Countly. In a real app, store prompt_id alongside the rendered message (or return it to your client) and read it back when the user rates the answer. Feedback is batched like interaction events; call feedback.flush() to send immediately, or feedback.shutdown() on process exit.

Full documentation

See the Countly AI SDK repository for the unified data model, observability levels, cost calculation, privacy controls, and Countly plugin integration (Drill, Funnels, Cohorts, APM, Crash Analytics).

License

MIT