ElevenLabs Plugin for OpenCode

Generate natural-sounding speech from text, clone voices, transcribe audio, and manage voices using ElevenLabs AI.

Features

• Text-to-Speech: Generate high-quality audio from text with customizable voices
• Voice Cloning: Create instant and professional voice clones from audio samples
• Speech-to-Text: Transcribe audio files to text
• Voice Management: List, inspect, and delete voices
• Model Information: Get details about available AI models
• User Information: Check subscription status and usage limits
• History: View generation history
• Voice Design: Create custom voices from descriptions

Prerequisites

• Node.js 18+ installed
• ElevenLabs API key (get one at https://elevenlabs.io/)

Setup

1. Install Dependencies

pnpm install

2. Configure Environment

cp .env.example .env
# Edit .env with your ElevenLabs API key

3. Get API Key

1. Sign up at https://elevenlabs.io/
2. Navigate to your profile settings
3. Copy your API key
4. Add it to your .env file:

ELEVENLABS_API_KEY=your_api_key_here

Using from Other Projects

Add to your opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": [
    "/absolute/path/to/elevenlabs/index.ts"
  ]
}

Available Tools

generate-audio

Generate natural-sounding speech from text.

Parameters:

• text (required): The text to convert to speech
• voiceId (optional): Voice ID to use (default: Rachel - "21m00Tcm4TlvDq8ikWAM")
• modelId (optional): Model ID for generation (default: "eleven_monolingual_v1")
• stability (optional): Voice stability 0-1 (default: 0.5)
• similarityBoost (optional): Similarity boost 0-1 (default: 0.75)
• outputPath (optional): Path to save the audio file

Examples:

# Basic text-to-speech
opencode run "generate audio from text 'Hello, world!'"

# With custom voice
opencode run "generate audio from text 'Welcome to ElevenLabs' using voice Rachel"

# Save to file
opencode run "generate audio from text 'Save this audio' and save to output.mp3"

# With custom settings
opencode run "generate audio with stability 0.8 and text 'Very stable voice'"

list-voices

List all available voices from ElevenLabs.

Parameters:

• showDetails (optional): Include detailed voice information (default: false)

Examples:

# List all voices
opencode run "list available voices"

# List with detailed information
opencode run "list voices with details"

get-voice

Get detailed information about a specific voice.

Parameters:

• voiceId (required): The ID of the voice to retrieve
• withSettings (optional): Include voice settings (default: true)

Examples:

# Get voice information
opencode run "get voice information for 21m00Tcm4TlvDq8ikWAM"

# Get Rachel's voice details
opencode run "get details for Rachel voice"

transcribe

Transcribe audio to text using ElevenLabs speech-to-text.

Parameters:

• audioPath (required): Path to audio file to transcribe
• language (optional): Language code (e.g., 'en', 'es', 'fr'). Auto-detected if not provided
• model (optional): Transcription model to use (default: "whisper-1")

Examples:

# Transcribe audio file
opencode run "transcribe audio from recording.mp3"

# Transcribe with specific language
opencode run "transcribe audio.wav in spanish"

# Transcribe with custom model
opencode run "transcribe audio.mp3 using whisper-1 model"

elevenlabs-recording-guidance

Get comprehensive guidance and best practices for recording audio samples for voice cloning.

Parameters:

• cloneType (optional): Type of cloning preparation (instant or professional)

Examples:

# Get recording guidance for instant cloning
opencode run "get recording guidance for voice cloning instant"

# Get guidance for professional cloning
opencode run "get recording guidance for professional voice cloning"

# Get all guidance
opencode run "get recording guidance for voice cloning"

elevenlabs-create-instant-voice-clone

Create an instant voice clone from audio files (1-5 minutes of good quality audio).

Parameters:

• voiceName (required): Name for the new cloned voice
• audioFiles (required): Array of file paths to audio files
• description (optional): Description of the voice

Examples:

# Create instant clone with single file
opencode run "create instant voice clone with name 'My Voice' and audio files ['sample.mp3']"

# Create instant clone with multiple files
opencode run "create instant voice clone with name 'Professional Voice' and audio files ['sample1.mp3', 'sample2.mp3', 'sample3.wav'] and description 'Business presentation voice'"

elevenlabs-create-professional-voice-clone

Start professional voice clone training (30-180 minutes of clean audio samples).

Parameters:

• voiceName (required): Name for the new professional voice clone
• audioFiles (required): Array of file paths to high-quality audio files
• description (optional): Description of the voice
• modelId (optional): Model to train on (default: "eleven_multilingual_v2")

Examples:

# Create professional voice clone
opencode run "create professional voice clone with name 'Audiobook Narrator' and audio files ['chapter1.mp3', 'chapter2.mp3'] and description 'Warm, engaging storytelling voice'"

# Create with custom model
opencode run "create professional voice clone with name 'Custom Voice' and audio files ['training1.wav', 'training2.wav'] using model eleven_multilingual_v2"

elevenlabs-get-clone-status

Check the training status and progress of a professional voice clone.

Parameters:

• voiceId (required): Voice ID to check status for

Examples:

# Check clone status
opencode run "check clone training status for voice ID abc123"

# Monitor professional clone progress
opencode run "get clone status for voice xyz789"

elevenlabs-add-clone-samples

Add additional audio samples to an existing voice clone to improve quality.

Parameters:

• voiceId (required): Voice ID to add samples to
• audioFiles (required): Array of file paths to additional audio files

Examples:

# Add more samples to voice
opencode run "add audio samples to voice abc123 with files ['new_sample1.mp3', 'new_sample2.wav']"

# Improve clone quality
opencode run "add clone samples to voice xyz789 with audio files ['improvement1.wav', 'improvement2.mp3']"

elevenlabs-delete-voice

Permanently delete a cloned voice from your ElevenLabs library.

Parameters:

• voiceId (required): Voice ID to delete

Examples:

# Delete a voice
opencode run "delete voice with ID abc123"

# Remove cloned voice
opencode run "permanently delete voice xyz789"

get-models

Get all available models from ElevenLabs.

Examples:

# List available models
opencode run "get available models"
opencode run "show me all elevenlabs models"

get-user-info

Get information about your account and subscription.

Examples:

# Check account info
opencode run "get my elevenlabs user info"
opencode run "check my subscription status"

get-history

Get history of generated audio items.

Parameters:

• pageSize (optional): Number of items per page (default: 10)

Examples:

# Get recent history
opencode run "get my generation history"

# Get more items
opencode run "get history with 20 items"

Common Use Cases

Generate Audio in Multiple Languages

# English
opencode run "generate audio from 'Hello, how are you?'"

# Spanish (use appropriate voice)
opencode run "generate audio from 'Hola, ¿cómo estás?' using Spanish voice"

Create Voice-Overs for Videos

opencode run "generate audio from 'Welcome to our tutorial on AI voice generation. In this video, we will explore the amazing capabilities of text-to-speech technology.' and save to voiceover.mp3"

Transcribe Meeting Recordings

opencode run "transcribe audio from meeting-recording.mp3"

Find the Perfect Voice

# List all voices
opencode run "list available voices"

# Get details about a specific voice
opencode run "get voice details for Antoni"

# Generate samples with different voices
opencode run "generate audio from 'Test voice' using voice Rachel"
opencode run "generate audio from 'Test voice' using voice Domi"

Voice Cloning Workflows

Instant Voice Cloning (Quick Results)

# 1. Get recording guidance
opencode run "get recording guidance for voice cloning instant"

# 2. Create instant clone
opencode run "create instant voice clone with name 'My Clone' and audio files ['sample1.mp3', 'sample2.mp3']"

# 3. Test the clone
opencode run "generate audio from 'Hello, this is my cloned voice!' using voice [CLONE_VOICE_ID]"

# 4. Clean up if needed
opencode run "delete voice with ID [CLONE_VOICE_ID]"

Professional Voice Cloning (Highest Quality)

# 1. Get professional recording guidance
opencode run "get recording guidance for voice cloning professional"

# 2. Create professional voice clone
opencode run "create professional voice clone with name 'Premium Voice' and audio files ['training1.wav', 'training2.wav', 'training3.wav'] and description 'Professional narration voice for audiobooks'"

# 3. Monitor training progress
opencode run "check clone training status for voice [CLONE_VOICE_ID]"

# 4. Add more samples to improve quality
opencode run "add more samples to voice [CLONE_VOICE_ID] with audio files ['additional1.wav', 'additional2.wav']"

# 5. Use professionally trained voice
opencode run "generate audio from 'This is my professionally trained voice speaking.' using voice [CLONE_VOICE_ID]"

Voice Cloning Overview

ElevenLabs offers two types of voice cloning:

Instant Voice Cloning (IVC)

• Requirements: 1-5 minutes of good quality audio
• Speed: Ready in seconds
• Quality: Good for quick prototyping and personal projects
• Best for: Content creators, podcasters, personal voice assistants

Professional Voice Cloning (PVC)

• Requirements: 30-180 minutes of clean audio samples
• Speed: Requires training time (minutes to hours)
• Quality: Highest fidelity with fine-tuning
• Best for: Audiobooks, professional voiceovers, enterprise applications

Voice IDs Reference

Common pre-built voice IDs:

• Rachel: 21m00Tcm4TlvDq8ikWAM (default, calm female)
• Domi: AZnzlk1XvdvUeBnXmlld (strong female)
• Bella: EXAVITQu4vr4xnSDxMaL (soft female)
• Antoni: ErXwobaYiN019PkySvjV (well-rounded male)
• Elli: MF3mGyEYCl7XYWbV9V6O (emotional female)
• Josh: TxGEqnHWrfWFTfGW9XjX (deep male)
• Arnold: VR6AewLTigWG4xSOukaG (crisp male)
• Adam: pNInz6obpgDQGcFmaJgB (deep male)
• Sam: yoZ06aMxZJJ28mfd3POQ (raspy male)

Cloned Voice IDs

• When you create a voice clone, it will be assigned a unique ID
• Use list-voices to see all voices including your clones
• Cloned voices appear in your voice library alongside pre-built voices

Use list-voices to see all available voices with their IDs.

Model IDs

Common model IDs:

• eleven_monolingual_v1: Best for English, fast generation
• eleven_multilingual_v1: Supports multiple languages
• eleven_multilingual_v2: Improved multilingual support

Use get-models to see all available models.

Configuration

Environment Variables

• ELEVENLABS_API_KEY (required): Your ElevenLabs API key
• ELEVENLABS_DEFAULT_VOICE_ID (optional): Default voice for generation
• ELEVENLABS_DEFAULT_MODEL_ID (optional): Default model for generation

Voice Settings

Customize voice generation with these parameters:

• Stability (0-1):

  • Lower (0-0.5): More variable and expressive
  • Higher (0.5-1): More stable and consistent

• Similarity Boost (0-1):

  • Lower (0-0.5): More creative interpretation
  • Higher (0.5-1): Closer to original voice

Voice Cloning Requirements

Audio Format

• Instant: MP3 192kbps+ (WAV provides little improvement)
• Professional: MP3 320kbps or WAV for best quality

Recording Environment

• Single speaker only: No background conversations
• Consistent volume: Avoid dramatic volume changes
• No background noise: Record in quiet environment
• Minimal room reverb: Avoid echo and reverberation
• No long silence gaps: Keep continuous speech flow

Content Recommendations

• Varying content: Different emotions and intonations
• Multiple contexts: Reading, conversation, narration
• Natural speech: Don't over-enunciate or act unnaturally
• Good microphone: Position 6-12 inches from mouth
• Duration:
  • Instant: 1-5 minutes total
  • Professional: 30-180 minutes total

Error Handling

Common errors and solutions:

API Key Missing

Error: ELEVENLABS_API_KEY environment variable is required

Solution: Add your API key to the .env file

Invalid Voice ID

Error: Voice not found

Solution: Use list-voices to find valid voice IDs

Audio File Not Found

Error: ENOENT: no such file or directory

Solution: Verify the audio file path is correct and the file exists

Rate Limit Exceeded

Error: Rate limit exceeded

Solution: Wait a moment or upgrade your ElevenLabs subscription

API Documentation

For more details about the ElevenLabs API:

• JavaScript SDK: https://github.com/elevenlabs/elevenlabs-js
• API Documentation: https://elevenlabs.io/docs/api-reference

Quickstart

Test the plugin:

pnpm run quickstart

This will list all available voices to verify your setup is working.

License

MIT