impactai-feedback-web

OpenTelemetry-based feedback tracking for LLM applications using standardized observability spans.

🎯 Features

• Simple feedback API - Intuitive methods for tracking user feedback
• OpenTelemetry powered - Uses industry-standard telemetry with OTLP export
• Browser & Node.js compatible - Works in both environments
• Auto-instrumentation - Captures user interactions, page loads, and network requests automatically
• TypeScript support - Full type definitions included
• Lightweight - Minimal bundle size and performance impact

🚀 Quick Start

Installation

npm install impactai-feedback-web @opentelemetry/api

Basic Usage

import ImpactWeb, { FeedbackStatus } from './node_modules/impactai-feedback-web/dist/index.esm.js';

// Initialize the feedback tracker
const impactWeb = new ImpactWeb({
  baseUrl: 'https://your-otlp-endpoint.com',
  publicKey: 'your-api-key',
  serviceName: 'my-llm-app'
});

// Send detailed feedback (traceId is required, name defaults to 'user-feedback')
await impactWeb.score({
  traceId: 'trace-abc-123',            // Required: Backend trace ID
  id: 'response-123',                  // Optional: Message ID
  value: 1,                            // Required: 1, -1, or 0
  comment: 'Very helpful response'     // Optional: Text feedback
});

// Or use the convenience method (traceId first, then optional params)
await impactWeb.feedback('trace-abc-123', 'response-456', FeedbackStatus.LIKED, 'Great answer!');

// Ensure spans are sent
await impactWeb.flush();

📋 API Reference

Constructor

const impactWeb = new ImpactWeb({
  baseUrl: string;           // Your OTLP endpoint (without /v1/traces)
  publicKey: string;         // Your API key
  serviceName?: string;      // Service name (default: 'llm-feedback-app')
  serviceVersion?: string;   // Service version (default: '1.0.0') 
  autoInstrumentation?: boolean; // Enable auto-instrumentation (default: true)
});

Methods

score(feedback: FeedbackScore): Promise<void>

Send detailed feedback with custom scoring:

await impactWeb.score({
  traceId: 'trace-abc-123',        // Required: Backend trace ID
  id: 'message-123',               // Optional: Unique identifier  
  name: 'quality-rating',          // Optional: Defaults to 'user-feedback'
  value: 1,                        // Required: 1 (LIKED), -1 (DISLIKED), 0 (NEUTRAL)
  comment: 'Helpful response',     // Optional: Text feedback
  metadata: {                      // Optional: Additional context
    messageType: 'chat',
    modelUsed: 'gpt-4'
  }
});

// Minimal usage (only required fields)
await impactWeb.score({
  traceId: 'trace-abc-123',
  value: 1
});

feedback(traceId: string, id?: string, status?: FeedbackStatus, comment?: string): Promise<void>

Simplified like/dislike feedback:

// Positive feedback (minimal)
await impactWeb.feedback('trace-abc-123');

// With all parameters
await impactWeb.feedback('trace-abc-123', 'msg-123', FeedbackStatus.LIKED, 'Great!');

// Negative feedback
await impactWeb.feedback('trace-abc-123', 'msg-456', FeedbackStatus.DISLIKED, 'Not helpful');

// Neutral feedback
await impactWeb.feedback('trace-abc-123', 'msg-789', FeedbackStatus.NEUTRAL, 'Okay response');

Feedback Values

| Value | FeedbackStatus | Meaning | |-------|----------------|---------| | 1 | FeedbackStatus.LIKED | Positive feedback | | -1 | FeedbackStatus.DISLIKED | Negative feedback | | 0 | FeedbackStatus.NEUTRAL | Neutral feedback |

event(name: string, properties?: Record<string, any>): Promise<void>

Track custom events:

await impactWeb.event('llm-query', {
  model: 'gpt-4',
  tokens: 150,
  responseTime: 1200
});

flush(): Promise<void>

Force export of pending spans:

await impactWeb.flush();

getSession(): { sessionId: string; initialized: boolean }

Get current session information:

const session = impactWeb.getSession();
console.log('Session ID:', session.sessionId);

🔧 Configuration

OTLP Endpoint

Your baseUrl should point to your OTLP collector endpoint without the /v1/traces path:

// ✅ Correct
const impactWeb = new ImpactWeb({
  baseUrl: 'https://your-endpoint.com',
  publicKey: 'your-api-key'
});

// ❌ Incorrect  
const impactWeb = new ImpactWeb({
  baseUrl: 'https://your-endpoint.com/v1/traces', // Don't include this
  publicKey: 'your-api-key'
});

Environment Variables

For security, use environment variables:

# .env.local
VITE_TELEMETRY_ENDPOINT=https://your-endpoint.com
VITE_TELEMETRY_API_KEY=your-secure-api-key

const impactWeb = new ImpactWeb({
  baseUrl: import.meta.env.VITE_TELEMETRY_ENDPOINT,
  publicKey: import.meta.env.VITE_TELEMETRY_API_KEY,
  serviceName: 'my-llm-app'
});

Auto-Instrumentation

By default, ImpactWeb enables automatic instrumentation in browsers that captures:

• User Interactions: All clicks, form submissions, keydown events
• Page Load Performance: Document load timing and metrics
• Network Requests: All fetch/XHR requests with timing

Auto-instrumentation is automatically disabled in Node.js environments.

Disable manually if needed:

const impactWeb = new ImpactWeb({
  baseUrl: 'https://your-endpoint.com',
  publicKey: 'your-api-key',
  autoInstrumentation: false
});

📊 Telemetry Data

Span Structure

Each feedback creates an OpenTelemetry span with rich attributes:

{
  "name": "feedback.user-feedback",
  "attributes": {
    "feedback.id": "message-123",
    "feedback.name": "user-feedback", 
    "feedback.value": 1,
    "feedback.comment": "Very helpful",
    "feedback.trace_id": "abc-def-123",
    "user.session_id": "session_1640995825_abc123",
    "user.agent": "Mozilla/5.0... (or 'server-side' in Node.js)",
    "app.name": "my-llm-app",
    "app.version": "1.0.0",
    "app.url": "https://myapp.com/chat (or 'server-side' in Node.js)"
  }
}

Integration with Backend

The spans include trace IDs for correlation with your backend:

// Frontend
await impactWeb.score({
  traceId: 'backend-trace-id',  // From your LLM API response
  value: 1
});

This allows you to correlate frontend feedback with backend LLM generation spans.

🎨 Framework Examples

React

import ImpactWeb, { FeedbackStatus } from './node_modules/impactai-feedback-web/dist/index.esm.js';

const impactWeb = new ImpactWeb({
  baseUrl: process.env.REACT_APP_TELEMETRY_ENDPOINT,
  publicKey: process.env.REACT_APP_TELEMETRY_API_KEY,
  serviceName: 'my-react-app'
});

function MessageComponent({ message, messageId, traceId }: Props) {
  const handleFeedback = async (status: FeedbackStatus) => {
    await impactWeb.feedback(traceId, messageId, status);
    await impactWeb.flush(); // Ensure spans are sent
  };

  return (
    <div>
      <p>{message.content}</p>
      <button onClick={() => handleFeedback(FeedbackStatus.LIKED)}>👍</button>
      <button onClick={() => handleFeedback(FeedbackStatus.DISLIKED)}>👎</button>
      <button onClick={() => handleFeedback(FeedbackStatus.NEUTRAL)}>😐</button>
    </div>
  );
}

Vue.js

<template>
  <div>
    <p>{{ message.content }}</p>
    <button @click="handleFeedback('LIKED')">👍</button>
    <button @click="handleFeedback('DISLIKED')">👎</button>
    <button @click="handleFeedback('NEUTRAL')">😐</button>
  </div>
</template>

<script setup lang="ts">
import ImpactWeb, { FeedbackStatus } from './node_modules/impactai-feedback-web/dist/index.esm.js';

const impactWeb = new ImpactWeb({
  baseUrl: import.meta.env.VITE_TELEMETRY_ENDPOINT,
  publicKey: import.meta.env.VITE_TELEMETRY_API_KEY
});

const handleFeedback = async (status: string) => {
  await impactWeb.feedback(
    props.traceId,           // Required first parameter
    props.messageId,         // Optional ID
    status as FeedbackStatus // Status
  );
  await impactWeb.flush();
};
</script>

Vanilla JavaScript (Browser)

<!DOCTYPE html>
<html>
<head>
    <title>LLM Feedback Example</title>
</head>
<body>
    <div id="message">AI response goes here...</div>
    <button onclick="sendFeedback(1)">👍 Like</button>
    <button onclick="sendFeedback(-1)">👎 Dislike</button>
    <button onclick="sendFeedback(0)">😐 Neutral</button>

    <script type="module">
        import ImpactWeb from './node_modules/impactai-feedback-web/dist/index.esm.js';

        const impactWeb = new ImpactWeb({
            baseUrl: 'https://your-endpoint.com',
            publicKey: 'your-api-key',
            serviceName: 'my-web-app'
        });

        window.sendFeedback = async (value) => {
            const traceId = 'trace-' + Date.now(); // Get this from your LLM API
            
            await impactWeb.score({
                traceId,                           // Required
                id: 'msg-' + Date.now(),          // Optional
                value: value,                      // 1, -1, or 0
                comment: value > 0 ? 'Liked' : value < 0 ? 'Disliked' : 'Neutral'
            });
            await impactWeb.flush();
            console.log('Feedback sent!');
        };
    </script>
</body>
</html>

Node.js

import ImpactWeb, { FeedbackStatus } from './node_modules/impactai-feedback-web/dist/index.esm.js';

const impactWeb = new ImpactWeb({
  baseUrl: 'https://your-endpoint.com',
  publicKey: 'your-api-key',
  serviceName: 'my-node-app'
});

// Send feedback from server-side
async function processFeedback(userId: string, responseId: string, traceId: string, rating: number) {
  await impactWeb.score({
    traceId,                           // Required: from your LLM generation
    id: responseId,                    // Optional: response identifier
    value: rating,                     // 1, -1, or 0
    metadata: {
      userId: userId,
      source: 'api'
    }
  });
  
  await impactWeb.flush();
  console.log('Server-side feedback recorded');
}

🔧 Advanced Usage

Custom Scoring with Different Metrics

// Quality rating (1-5 scale, but normalized to -1, 0, 1)
await impactWeb.score({
  traceId: 'trace-abc-123',
  name: 'quality-rating',
  value: rating >= 4 ? 1 : rating <= 2 ? -1 : 0,  // Convert 5-point to 3-point scale
  comment: 'Good but could be more detailed',
  metadata: {
    originalRating: rating,              // Keep original 1-5 rating
    category: 'technical-question',
    difficulty: 'intermediate'
  }
});

// Binary classification
await impactWeb.score({
  traceId: 'trace-abc-123',
  name: 'factual-accuracy',
  value: isAccurate ? 1 : -1,           // Only positive or negative
  metadata: {
    verifiedBy: 'expert-review'
  }
});

Event Tracking

// Track LLM interactions
await impactWeb.event('llm-query-start', {
  model: 'gpt-4-turbo',
  temperature: 0.7,
  maxTokens: 1000
});

// Track user actions
await impactWeb.event('feature-used', {
  feature: 'code-generation',
  language: 'python'
});

📋 FeedbackStatus Enum

enum FeedbackStatus {
  LIKED = 'LIKED',      // Maps to value: 1
  DISLIKED = 'DISLIKED', // Maps to value: -1
  NEUTRAL = 'NEUTRAL'    // Maps to value: 0
}

🐛 Troubleshooting

Common Issues

1. Import errors: Use the direct ESM path: ./node_modules/impactai-feedback-web/dist/index.esm.js
2. Missing traceId: traceId is now required - get it from your LLM API response
3. Spans not appearing: Check endpoint URL (should not include /v1/traces) and API key
4. CORS errors: Ensure your endpoint accepts requests from your domain
5. TypeScript errors: Make sure @opentelemetry/api is installed as peer dependency

API Changes in v1.2.0

• Breaking: traceId is now required in score() method
• Breaking: feedback() method parameter order changed: feedback(traceId, id?, status?, comment?)
• Improvement: name parameter in score() is now optional (defaults to 'user-feedback')
• Improvement: Better TypeScript documentation for feedback values

Testing Your Integration

// Test the integration with required traceId
await impactWeb.score({
  traceId: 'test-trace-' + Date.now(),
  id: 'test-' + Date.now(),
  value: 1,
  comment: 'Testing ImpactWeb integration'
});

await impactWeb.flush();
console.log('Test feedback sent!');

Debug Output

The package logs its operations to the console:

🔧 Initializing ImpactWeb...
✅ ImpactWeb initialized successfully
📊 Recording feedback score: user-feedback = 1
✅ Feedback score recorded: user-feedback = 1
📤 Exporting 1 spans to https://your-endpoint.com
✅ Spans exported successfully

📊 Key Benefits

• Industry Standard: Uses OpenTelemetry Protocol (OTLP) for maximum compatibility
• Flexible Backend: Works with any OTLP-compatible endpoint (Jaeger, Zipkin, etc.)
• Rich Context: Automatic instrumentation captures user interactions and performance
• Type Safe: Full TypeScript support with comprehensive type definitions
• Cross-Platform: Works in browsers and Node.js environments
• Correlation: Links frontend feedback with backend traces via required trace IDs

📄 License

MIT License - feel free to use in your projects.

🤝 Contributing

Issues and pull requests welcome! This package aims to provide a standardized, OpenTelemetry-based approach to LLM feedback tracking.