@ph0ny/sdk

TypeScript SDK for the ph0ny Voice API. Build voice agents with speech-to-text, text-to-speech, RAG knowledge bases, and conversational AI — all through one client.

Installation

npm install @ph0ny/sdk

Quick Start

import { createVoiceClient } from '@ph0ny/sdk';

const client = createVoiceClient({
  apiKey: 'plabs_...',
});

// Create an agent with a knowledge base in one call
const { agent, collection } = await client.agents.provision({
  name: 'Support Agent',
  systemPrompt: 'You are a helpful support agent for Acme Corp.',
  voiceId: 'alloy',
  llmProvider: 'openai',
  collection: { name: 'Support Docs' },
  initialContent: [
    { content: 'Return policy: 30-day returns on all items...' },
    { content: 'Shipping: Free shipping on orders over $50...' },
  ],
});

// Chat with the agent (text, RAG-grounded)
const response = await client.agents.chat(agent.id, {
  message: 'What is the return policy?',
});
console.log(response.message);

// Have the agent speak
const audio = await client.agents.speak(agent.id, {
  text: response.message,
});
console.log(audio.audioUrl);

Client Setup

import { createVoiceClient } from '@ph0ny/sdk';

const client = createVoiceClient({
  apiKey: 'plabs_...',           // Required — get from developer portal
  baseUrl: 'https://...',        // Optional — defaults to production API
  timeout: 30000,                // Optional — request timeout in ms

  // Bring Your Own Key (BYOK) — use your own provider API keys
  // Requests using BYOK keys bypass platform usage metering
  providerKeys: {
    elevenlabs: 'sk-...',        // ElevenLabs TTS/STT
    openai: 'sk-...',            // OpenAI LLM
    anthropic: 'sk-ant-...',     // Anthropic LLM
    cartesia: 'sk-cart-...',     // Cartesia TTS/STT
    deepgram: 'sk-dg-...',       // Deepgram STT
    fishAudio: 'sk-fish-...',    // Fish Audio TTS
    inworld: 'sk-inw-...',       // Inworld TTS key
    inworldSecret: 'secret-...', // Inworld TTS secret
    resembleAi: 'sk-res-...',    // Resemble AI TTS
    huggingFace: 'hf_...',       // Hugging Face (qwen-tts)
  },
});

Agents

Create, manage, and interact with voice agents. Each agent combines an LLM, a voice, and an optional knowledge base.

// Create an agent
const agent = await client.agents.create({
  name: 'Sales Assistant',
  systemPrompt: 'You help customers find the right product.',
  firstMessage: 'Hi! How can I help you today?',
  voiceId: 'alloy',
  ttsProvider: 'default',       // 'default' | 'elevenlabs' | 'pocket-tts'
  llmProvider: 'openai',        // 'openai' | 'anthropic'
  llmModel: 'gpt-4o',
  language: 'en',
  temperature: 0.7,
  collectionId: 'col_...',      // Link to a RAG collection
});

// One-shot provision: create agent + collection + ingest content
const result = await client.agents.provision({
  name: 'Product Expert',
  systemPrompt: '...',
  collection: { name: 'Product Catalog' },
  initialContent: [
    { content: '...', contentType: 'text' },
    { content: '...', contentType: 'transcript' },
  ],
});
// result.agent, result.collection, result.ingested

// CRUD
const agents = await client.agents.listAll();
const agent = await client.agents.get('agent_...');
await client.agents.update('agent_...', { temperature: 0.5 });
await client.agents.delete('agent_...');

// Chat (text conversation with RAG context)
const chat = await client.agents.chat('agent_...', {
  message: 'How do I reset my password?',
  conversationId: 'conv_...',   // Optional — continues a conversation
});
// chat.message, chat.ragContext, chat.usage

// Speak (synthesize text with the agent's configured voice)
const speech = await client.agents.speak('agent_...', {
  text: 'Welcome! How can I help?',
  format: 'mp3',
});
// speech.audioUrl or speech.audio (base64)

Speech-to-Text

Transcribe audio from files, buffers, or URLs. Supports sync and async modes.

// Sync transcription (blocks until complete)
const result = await client.stt.transcribe('https://example.com/call.mp3', {
  mode: 'accurate',            // 'fast' | 'accurate'
  timestamps: 'word',          // 'none' | 'word' | 'segment'
  diarize: true,               // Speaker diarization
  language: 'en',
});
// result.text, result.segments, result.words, result.duration

// From a file buffer
const result = await client.stt.transcribe(audioBuffer);

// Async transcription (for large files)
const job = await client.stt.transcribeAsync(audioBuffer, { mode: 'accurate' });
const result = await client.stt.waitForJob(job.id, {
  pollInterval: 1000,
  timeout: 300000,
});

// Or poll manually
const status = await client.stt.getJob(job.id);
if (status.status === 'completed') {
  console.log(status.result.text);
}

// Mistral provider (direct, uses your Mistral API key)
const client = createVoiceClient({
  apiKey: 'plabs_...',
  mistralApiKey: 'your-mistral-key',
});
const result = await client.stt.transcribe(audio, {
  provider: 'mistral',
  context_bias: ['Acme Corp', 'HIPAA'],  // Spelling hints
});

Text-to-Speech

Synthesize speech from text. Multiple voices and providers available.

// Synthesize
const result = await client.tts.synthesize('Hello, welcome to Acme!', {
  voiceId: 'alloy',
  format: 'mp3',               // 'mp3' | 'wav' | 'ogg'
  speed: 1.0,                  // 0.5 - 2.0
  pitch: 1.0,                  // 0.5 - 2.0
});
// result.audioUrl or result.audio (base64), result.duration, result.charactersUsed

// List voices
const voices = await client.tts.listAllVoices({ includeCustom: true });
// Or paginated:
const page = await client.tts.listVoices({ limit: 20, cursor: '...' });

// Get a specific voice
const voice = await client.tts.getVoice('voice_...');

// Clone a voice from audio samples
const cloned = await client.tts.cloneVoice(
  [audioSample1, audioSample2],     // Blob[] or Buffer[]
  { name: 'My Custom Voice', description: 'Warm, friendly tone' }
);
// cloned.voice.id, cloned.quality

// Delete a custom voice
await client.tts.deleteVoice('voice_...');

RAG (Knowledge Bases)

Create collections, ingest content, and search with hybrid vector + keyword retrieval.

// Create a collection
const collection = await client.rag.createCollection({
  name: 'Company Wiki',
  description: 'Internal knowledge base',
});

// Ingest text content
const doc = await client.rag.ingest(collection.id, {
  content: 'Our return policy allows 30-day returns...',
  contentType: 'text',         // 'text' | 'transcript' | 'audio' | 'url'
  chunkSize: 1000,             // Characters per chunk (100-4000)
  chunkOverlap: 200,           // Overlap between chunks (0-500)
});
// doc.documentId, doc.chunksCreated, doc.tokensUsed

// Ingest from URL
await client.rag.ingest(collection.id, {
  sourceUrl: 'https://example.com/docs/faq',
  contentType: 'url',
});

// Ingest audio (transcribes then ingests)
await client.rag.ingestAudio(collection.id, audioFile);

// Search
const results = await client.rag.search(collection.id, {
  query: 'return policy',
  limit: 10,                   // 1-100
  threshold: 0.7,              // Minimum relevance (0-1)
  includeMetadata: true,
});
// results.results[].content, .score, .metadata

// Manage collections
const collections = await client.rag.listAllCollections();
const col = await client.rag.getCollection('col_...');
await client.rag.deleteDocument('col_...', 'doc_...');
await client.rag.deleteCollection('col_...');

Usage & Billing

const usage = await client.getUsage();
console.log(`STT: ${usage.stt.minutesUsed}/${usage.stt.minutesLimit} min`);
console.log(`TTS: ${usage.tts.charactersUsed}/${usage.tts.charactersLimit} chars`);
console.log(`RAG: ${usage.rag.queriesUsed}/${usage.rag.queriesLimit} queries`);

Usage headers are returned with every API response:

| Header | Description | |--------|-------------| | X-Usage-Current | Current usage for this billing period | | X-Usage-Limit | Maximum allowed for your tier | | X-Usage-Remaining | Remaining quota | | X-Usage-Bypassed | Whether BYOK bypassed metering |

Pricing Tiers

| Tier | STT | TTS | RAG | |------|-----|-----|-----| | Free | 60 min/mo | 10K chars/mo | 100 queries/mo | | Starter | 600 min/mo | 100K chars/mo | 1K queries/mo | | Pro | 6,000 min/mo | 1M chars/mo | 10K queries/mo |

Error Handling

All API errors throw VoiceApiException with structured error information:

import { VoiceApiException } from '@ph0ny/sdk';

try {
  await client.stt.transcribe(audio);
} catch (err) {
  if (err instanceof VoiceApiException) {
    console.error(err.code);       // 'RATE_LIMITED', 'QUOTA_EXCEEDED', etc.
    console.error(err.message);    // Human-readable message
    console.error(err.statusCode); // HTTP status code
    console.error(err.details);    // Additional context
  }
}

BYOK (Bring Your Own Key)

Pass your own provider API keys to use providers directly. BYOK requests bypass platform usage metering — you pay the provider directly.

const client = createVoiceClient({
  apiKey: 'plabs_...',
  providerKeys: {
    elevenlabs: 'sk-...',
    openai: 'sk-...',
    anthropic: 'sk-ant-...',
    cartesia: 'sk-cart-...',
    deepgram: 'sk-dg-...',
    fishAudio: 'sk-fish-...',
    inworld: 'sk-inw-...',
    inworldSecret: 'secret-...',
    resembleAi: 'sk-res-...',
    huggingFace: 'hf_...',
  },
});

// This request uses your ElevenLabs key directly
await client.tts.synthesize('Hello', {
  voiceId: 'custom-voice',
  provider: 'elevenlabs',
});

Types

All request/response types are exported and validated with Zod:

import type {
  // Client
  VoiceClientConfig,
  ProviderKeys,

  // STT
  TranscribeOptions,
  TranscribeResult,
  TranscribeJob,
  TranscriptSegment,

  // TTS
  SynthesizeOptions,
  SynthesizeResult,
  Voice,
  CloneVoiceOptions,
  CloneVoiceResult,

  // RAG
  Collection,
  IngestOptions,
  IngestResult,
  SearchOptions,
  SearchResponse,
  SearchResult,

  // Agents
  Agent,
  CreateAgentOptions,
  UpdateAgentOptions,
  ProvisionAgentOptions,
  ProvisionAgentResult,
  ChatOptions,
  ChatResponse,
  SpeakOptions,
  SpeakResult,

  // Common
  UsageStats,
  VoiceApiException,
  PaginatedResponse,
  PaginationOptions,
} from '@ph0ny/sdk';

License

MIT