OpenInference Vercel

This package provides utilities to ingest Vercel AI SDK spans into platforms like Arize and Phoenix.

Note: This package targets AI SDK v6 and is tested against v6 telemetry. Older versions (>= 3.3) are best-effort compatible.

AI SDK Compatibility

| AI SDK version | Support level | Notes | | -------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | v6.x | Targeted | Emits gen_ai.* (OTel GenAI semconv) + ai.* (Vercel-specific). @arizeai/openinference-vercel prefers gen_ai.* and falls back to ai.*. | | v5.x | Best effort | Telemetry primarily uses ai.*. Some standard gen_ai.*-derived mappings may be unavailable. | | >= 3.3 and < 5 | Best effort | Telemetry is experimental; attribute shapes may differ. |

Installation

npm install --save @arizeai/openinference-vercel

You will also need to install OpenTelemetry and Vercel packages to your project.

npm i @opentelemetry/api @vercel/otel @opentelemetry/exporter-trace-otlp-proto @arizeai/openinference-semantic-conventions

Usage

@arizeai/openinference-vercel provides a set of utilities to help you ingest Vercel AI SDK spans into platforms and works in conjunction with Vercel's OpenTelemetry support. To get started, you will need to add OpenTelemetry support to your Vercel project according to their guide

To process your Vercel AI SDK Spans add a OpenInferenceSimpleSpanProcessor or OpenInferenceBatchSpanProcessor to your OpenTelemetry configuration.

[!NOTE] The OpenInferenceSpanProcessor does not handle the exporting of spans so you will pass it an exporter as a parameter.

// instrumentation.ts
import { registerOTel } from "@vercel/otel";
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
import {
  isOpenInferenceSpan,
  OpenInferenceSimpleSpanProcessor,
} from "@arizeai/openinference-vercel";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";

// For troubleshooting, set the log level to DiagLogLevel.DEBUG
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

export function register() {
  registerOTel({
    serviceName: "phoenix-next-app",
    attributes: {
      // This is not required but it will ensure your traces get added to a specific project in Arize Phoenix
      [SEMRESATTRS_PROJECT_NAME]: "your-next-app",
    },
    spanProcessors: [
      new OpenInferenceSimpleSpanProcessor({
        exporter: new OTLPTraceExporter({
          headers: {
            // API key if you are sending it to Phoenix Cloud
            api_key: process.env["PHOENIX_API_KEY"] || "",
            // API key if you are sending it to local Phoenix
            Authorization: `Bearer ${process.env["PHOENIX_API_KEY"]}` || "",
          },
          url:
            process.env["PHOENIX_COLLECTOR_ENDPOINT"] ||
            "https://app.phoenix.arize.com/v1/traces",
        }),
        spanFilter: (span) => {
          // Only export spans that are OpenInference to negate non-generative spans
          // This should be removed if you want to export all spans
          return isOpenInferenceSpan(span);
        },
      }),
    ],
  });
}

Now enable telemetry in your AI SDK calls by setting the experimental_telemetry parameter to true.

const result = await generateText({
  model: openai("gpt-4-turbo"),
  prompt: "Write a short story about a cat.",
  experimental_telemetry: { isEnabled: true },
});

For details on Vercel AI SDK telemetry see the Vercel AI SDK Telemetry documentation.

Examples

To see an example go to the Next.js OpenAI Telemetry Example in the examples directory of this repo.

For more information on Vercel OpenTelemetry support see the Vercel AI SDK Telemetry documentation.