LiveTalking Streaming SDK

A Node.js SDK for streaming avatars using LiveTalking with Wav2Lip technology, fully compatible with your enhanced streaming API backend.

Installation

npm install @jona27/livetalking-streaming-sdk

Quick Start

const { StreamingAvatar } = require('@jona27/livetalking-streaming-sdk');

async function main() {
  // Initialize the streaming avatar
  const avatar = new StreamingAvatar({
    serverUrl: 'http://localhost:8010', // Your LiveTalking server URL
    debug: true
  });

  try {
    // Create authentication token
    await avatar.createToken({ user_id: 'your_user_id' });
    
    // Create and start avatar session
    const sessionData = await avatar.createStartAvatar({
      avatarId: 'wav2lip256_avatar1', // Your avatar ID
      quality: 'medium'
    });
    
    console.log('Session created:', sessionData.sessionId);

    // Set up event listeners
    avatar.on('avatar_start_talking', (text) => {
      console.log('Avatar started talking:', text);
    });

    avatar.on('stream_ready', (data) => {
      console.log('Stream ready:', data.sessionId);
    });

    // Make avatar speak
    await avatar.speak({
      text: 'Hello! This is your streaming avatar.',
      interrupt: false
    });

    // Interrupt current speech
    await avatar.interrupt();

    // Close session when done
    await avatar.closeAvatar();
    
  } catch (error) {
    console.error('Error:', error.message);
  }
}

main();

API Reference

StreamingAvatar

Constructor

new StreamingAvatar(config)

Parameters:

• config.serverUrl (string): Your LiveTalking server URL
• config.token (string, optional): Pre-existing authentication token
• config.debug (boolean, optional): Enable debug logging

Methods

createToken(request?)

Creates an authentication token for API access.

Parameters:

• request.user_id (string, optional): User identifier

Returns: Promise

createStartAvatar(request)

Creates and starts an avatar session with WebSocket connection.

Parameters:

• request.avatarId (string): Avatar ID (e.g., 'wav2lip256_avatar1')
• request.quality ('low' | 'medium' | 'high', optional): Video quality
• request.voice (object, optional): Voice configuration
• request.knowledgeBase (string, optional): Knowledge base for LLM

Returns: Promise

startSession()

Starts an existing session (if not auto-started).

speak(request)

Makes the avatar speak text via WebSocket or HTTP.

Parameters:

• request.text (string): Text to speak
• request.interrupt (boolean, optional): Whether to interrupt current speech

interrupt()

Interrupts current avatar speech.

closeAvatar()

Closes the avatar session and cleans up resources.

getSessionId()

Gets the current session ID.

Returns: string | null

isAvatarConnected()

Checks if avatar is connected via WebSocket.

Returns: boolean

sendWebRTCOffer(sdp)

Sends WebRTC offer for direct video streaming.

Parameters:

• sdp (string): WebRTC SDP offer

ping()

Sends ping to keep WebSocket connection alive.

Events

• avatar_start_talking: Fired when avatar starts speaking with text
• avatar_stop_talking: Fired when avatar stops speaking
• stream_ready: Fired when stream is ready with session data
• stream_disconnected: Fired when stream disconnects
• error: Fired when an error occurs
• connection_established: Fired when WebRTC connection is established
• session_created: Fired when session is created with session ID
• websocket_connected: Fired when WebSocket connects
• websocket_disconnected: Fired when WebSocket disconnects

Enhanced API Integration

This SDK integrates with your enhanced streaming API backend:

Authentication Flow

1. Token Creation: POST /api/v1/streaming/create_token
2. Session Creation: POST /api/v1/streaming/new
3. WebSocket Connection: GET /ws/sdk/{session_id}

Real-time Communication

• WebSocket: Real-time bidirectional communication
• HTTP Fallback: API endpoints for speak/interrupt
• Event System: Real-time status updates

Available Endpoints

• POST /api/v1/streaming/create_token - Create auth token
• POST /api/v1/streaming/new - Create session
• POST /api/v1/streaming/start - Start session
• DELETE /api/v1/streaming/end_session - End session
• POST /api/v1/streaming/speak - Make avatar speak
• POST /api/v1/streaming/interrupt - Interrupt speech
• GET /api/v1/streaming/avatars - List available avatars
• GET /api/v1/streaming/health - Health check

Backend Requirements

Make sure your LiveTalking server is running with the enhanced API:

python app.py --model wav2lip --transport webrtc --listenport 8010

The enhanced API will be available at: http://localhost:8010/api/v1/streaming/

Features

• ✅ JWT-based authentication
• ✅ WebSocket real-time communication
• ✅ HTTP API fallback
• ✅ Session management
• ✅ Text-to-speech with Wav2Lip
• ✅ Speech interruption
• ✅ Event-driven architecture
• ✅ TypeScript support
• ✅ WebRTC integration ready
• ✅ Keep-alive ping/pong
• ✅ Error handling and recovery

License

Apache-2.0