Senpai Service Library

A reusable TypeScript library for capturing audio/video and performing real-time transcription and mentoring.

Features

• Audio and video capture using the MediaStream API
• Real-time audio processing using AudioWorklet (embedded, no external files needed)
• WebSocket-based Senpai Service integration
• Full TypeScript support with type definitions
• Configurable options and callbacks
• ES Module format for modern JavaScript applications

Installation

# If you're using npm
npm install senpai-service

# If you're using yarn
yarn add senpai-service

Or include it directly in your HTML:

<script type="module">
    import SenpaiService from './path/to/senpai-service.js';
    // Your code here
</script>

Usage

Basic Usage

import SenpaiService from 'senpai-service';

// Get DOM elements
const localVideoElement = document.getElementById("local-video") as HTMLVideoElement;
const remoteVideoElement = document.getElementById("remote-video") as HTMLVideoElement;
const transcriptionElement = document.getElementById("transcription");

// Create an instance of the SenpaiService
const senpaiService = new SenpaiService({
    webSocketUrl: "wss://your-service.url/ws",
    // Callback for transcription updates
    onTranscriptionUpdate: (message) => {
        transcriptionElement.innerText += `${message.text}\n`;
    },
    onInterimTranscriptionUpdate: (message) => {
        console.log("Interim:", message.text);
    },
    onNudge: (message) => {
        console.log("Nudge:", message.text);
    },
    onStatusChange: (status) => {
        console.log("Status:", status.status);
    }
});

// Start recording with video
document.getElementById("start-button")?.addEventListener("click", async () => {
    await senpaiService.start(localVideoElement, remoteVideoElement);
});

// Stop recording
document.getElementById("stop-button")?.addEventListener("click", async () => {
    await senpaiService.stop();
});

// Check if recording
console.log("Is active:", senpaiService.isActive());

Configuration Options

The SenpaiService constructor accepts an options object with the following properties:

| Option | Type | Default | Description | |--------------------------------|----------------------------------------------|--------------------|-----------------------------------------------------------------------| | webSocketUrl | string | "ws://localhost:8080/ws" | URL of the WebSocket Senpai Service | | prompt | string | "" | System prompt (development only) | | textOnly | boolean | false | Enable text-only mode | | languages | string[] | undefined | Array of language codes for transcription | | onTranscriptionUpdate | (message: TranscriptionMessage) => void | () => {} | Callback for final transcription updates | | onInterimTranscriptionUpdate | (message: TranscriptionMessage) => void | () => {} | Callback for interim transcription updates | | onNudge | (message: TranscriptionMessage) => void | () => {} | Callback for nudge messages | | onError | (message: string, error?: any) => void | console.error | Callback function for error handling | | onStatusChange | (status: StatusUpdate) => void | () => {} | Callback for status changes (connected, recording, stopped, etc.) | | debugging | boolean | false | Enable debug logging |

API Reference

Methods

• start(localVideoElement, remoteVideoElement): Starts audio/video capture and transcription

  • localVideoElement: HTMLVideoElement for the local user's video
  • remoteVideoElement: HTMLVideoElement for the remote user's video
  • Returns: Promise<void>

• stop(): Stops transcription and cleans up resources

  • Returns: Promise<void>

• isActive(): Checks if the service is currently recording

  • Returns: boolean indicating recording state

• sendText(text): Send text message through WebSocket (for testing)

  • text: string - Text to send
  • Returns: void

Events

The library provides callbacks for various events:

• onTranscriptionUpdate(message): Called when new final transcription is received
  • message.type: 'final'
  • message.text: The transcription text
• onInterimTranscriptionUpdate(message): Called when interim transcription is received
  • message.type: 'interim'
  • message.text: The interim transcription text
• onNudge(message): Called when a nudge message is received
  • message.type: 'nudge'
  • message.text: The nudge message
• onError(message, error): Called when an error occurs
• onStatusChange(status): Called when the status changes
  • status.status: 'connected' | 'disconnected' | 'recording' | 'stopped' | 'error'
  • status.error: Error object (only present when status is 'error')

Dependencies

• Modern browser with support for:
  • MediaStream API
  • AudioWorklet API
  • WebSocket API
  • ES Modules

TypeScript Support

The library is written in TypeScript and includes full type definitions. Key interfaces:

interface SenpaiServiceOptions {
    webSocketUrl?: string;
    prompt?: string;
    textOnly?: boolean;
    languages?: string[];
    onInterimTranscriptionUpdate?: (message: TranscriptionMessage) => void;
    onTranscriptionUpdate?: (message: TranscriptionMessage) => void;
    onNudge?: (message: TranscriptionMessage) => void;
    onError?: (message: string, error?: any) => void;
    onStatusChange?: (status: StatusUpdate) => void;
    debugging?: boolean;
}

interface TranscriptionMessage {
    type: 'interim' | 'final' | 'nudge';
    text: string;
}

interface StatusUpdate {
    status: 'recording' | 'connected' | 'disconnected' | 'stopped' | 'error';
    error?: any;
}

License

MIT