use-voice-control

React hooks and components for seamless voice control and speech I/O (Speech-to-Text and Text-to-Speech). Build voice-enabled applications with client-side speech recognition and server-side or client-side speech synthesis.

npm install use-voice-control

🎤 Features

Speech-to-Text (STT)

• Moonshine.js: Fast, accurate client-side speech recognition running entirely in the browser
• WebRTC Audio Capture: Real-time microphone input with automatic gain control
• Multi-language Support: Recognize speech in 100+ languages
• Offline-First: No external API calls required for STT

Text-to-Speech (TTS)

• Kokoro TTS: Natural-sounding synthesis on Node.js/Edge with 16 pre-trained voices
• Deepgram TTS: Enterprise-grade speech synthesis with 12 Aura voices
• Server & Client Support: Run on backend or stream to client
• Multiple Output Formats: WAV, PCM, or raw audio buffers

React Integration

• Custom Hooks: useVoiceControl(), useSpeechRecognition(), useSpeechSynthesis()
• Pre-built Components: Audio recorder, voice selector, playback controls
• State Management: Streaming, loading, error handling baked in

🚀 Quick Start

Installation

npm install use-voice-control

Basic STT Example

import { useSpeechRecognition } from 'use-voice-control/hooks';

export function VoiceInput() {
  const { isListening, transcript, startListening, stopListening } = useSpeechRecognition({
    language: 'en-US'
  });

  return (
    <div>
      <button onClick={startListening} disabled={isListening}>
        🎤 Start Recording
      </button>
      <button onClick={stopListening} disabled={!isListening}>
        ⏹️ Stop
      </button>
      <p>You said: {transcript}</p>
    </div>
  );
}

Basic TTS Example

import { useSpeechSynthesis } from 'use-voice-control/hooks';

export function VoiceOutput() {
  const { speak, isSpeaking } = useSpeechSynthesis({
    provider: 'kokoro',
    voice: 'af_heart' // Female voice
  });

  return (
    <button onClick={() => speak("Hello, world!")}>
      {isSpeaking ? '🔊 Speaking...' : '▶️ Play'}
    </button>
  );
}

📚 API Reference

Hooks

useSpeechRecognition(options)

Captures audio from the user's microphone and converts it to text using Moonshine.js.

Options:

interface SpeechRecognitionOptions {
  language?: string;           // Default: 'en-US'
  autoStart?: boolean;          // Default: false
  onTranscript?: (text: string) => void;
  onError?: (error: Error) => void;
  autoStopTimeout?: number;     // Auto-stop after silence (ms)
}

Returns:

{
  transcript: string;           // Current transcribed text
  isListening: boolean;         // Recording in progress
  isFinal: boolean;            // Transcript is final
  startListening: () => void;
  stopListening: () => void;
  resetTranscript: () => void;
  error: Error | null;
}

useSpeechSynthesis(options)

Converts text to speech and plays it back with optional streaming.

Options:

interface SpeechSynthesisOptions {
  provider?: 'kokoro' | 'deepgram';  // Default: 'kokoro'
  voice?: string;                     // Provider-specific voice ID
  rate?: number;                      // Speech rate (0.5 - 2.0)
  pitch?: number;                     // Voice pitch (0.5 - 2.0)
  volume?: number;                    // Volume (0 - 1)
  onError?: (error: Error) => void;
}

Returns:

{
  speak: (text: string) => Promise<void>;
  pause: () => void;
  resume: () => void;
  stop: () => void;
  isSpeaking: boolean;
  isPaused: boolean;
  currentTime: number;
  duration: number;
  error: Error | null;
}

useVoiceControl(options)

Combined STT + TTS hook for full voice control workflows.

Options:

interface VoiceControlOptions {
  sttOptions?: SpeechRecognitionOptions;
  ttsOptions?: SpeechSynthesisOptions;
  onTranscriptEnd?: (transcript: string) => Promise<string>;
  autoPlayResponse?: boolean;  // Auto-play TTS response
}

Returns:

{
  // STT state
  transcript: string;
  isListening: boolean;
  // TTS state
  isSpeaking: boolean;
  // Combined control
  toggleListening: () => void;
  speak: (text: string) => void;
  reset: () => void;
}

Voice Options

Kokoro Voices

16 professional voices optimized for natural speech synthesis:

Female Voices:

af_heart    - Warm, caring tone
af_alloy    - Neutral, professional
af_aoede    - Bright, energetic
af_bella    - Soft, gentle
af_jessica  - Friendly, conversational
af_nicole   - Clear, articulate
af_river    - Calm, soothing
af_sarah    - Warm, approachable
af_sky      - Young, vibrant

Male Voices:

am_adam     - Deep, authoritative
am_echo     - Resonant, smooth
am_fable    - Narrative, engaging
am_fenrir   - Bold, strong
am_liam     - Friendly, warm
am_michael  - Professional, clear
am_onyx     - Dark, mysterious

Example:

const { speak } = useSpeechSynthesis({
  provider: 'kokoro',
  voice: 'af_heart'
});

Deepgram Aura Voices

12 natural-sounding voices for enterprise applications:

angus, asteria, arcas, orion, orpheus, athena,
luna, zeus, perseus, helios, hera, stella

Example:

const { speak } = useSpeechSynthesis({
  provider: 'deepgram',
  voice: 'luna'
});

Components

<AudioRecorder />

Pre-built recording interface with visual feedback.

import { AudioRecorder } from 'use-voice-control/components';

<AudioRecorder 
  onTranscript={(text) => console.log(text)}
  language="en-US"
/>

<VoiceSelector />

Dropdown to choose between available voices.

import { VoiceSelector } from 'use-voice-control/components';

<VoiceSelector 
  provider="kokoro"
  onChange={(voice) => setSelectedVoice(voice)}
/>

<AudioPlayer />

Controls for playback of generated speech.

import { AudioPlayer } from 'use-voice-control/components';

<AudioPlayer 
  src={audioUrl}
  autoPlay={false}
/>

🔌 Core API

Server-Side Speech Synthesis

Generate speech audio directly from the server or Edge runtime:

import { generateSpeech } from 'use-voice-control/speech';

// Generate Kokoro speech
const audio = await generateSpeech({
  text: "Hello, world!",
  provider: 'kokoro',
  voice: 'af_heart'
});

// Returns: { audio: ArrayBuffer, contentType: string }

TypeScript Support

Full type definitions included:

import type { TTSProvider, KokoroVoice, DeepgramSpeaker } from 'use-voice-control/speech';

const provider: TTSProvider = 'kokoro';
const voice: KokoroVoice = 'af_heart';

🏗️ Architecture

Speech-to-Text Pipeline

1. Audio Capture → WebRTC microphone input with automatic gain control
2. Buffering → Circular audio buffer with silence detection
3. Inference → Moonshine.js runs model in Web Worker to avoid blocking
4. Streaming → Real-time transcript updates as user speaks
5. Final Output → Complete transcript on stop or timeout

Text-to-Speech Pipeline

1. Input Processing → Text validation and segmentation
2. Synthesis → Kokoro or Deepgram provider synthesis
3. Format Conversion → Audio buffer to playable format
4. Streaming → Optional chunked playback
5. Playback Control → Native audio element with pause/resume/volume

🎯 Use Cases

Customer Support Chatbots

<VoiceControl 
  onTranscriptEnd={async (text) => {
    const response = await fetchChatbotResponse(text);
    return response;
  }}
  autoPlayResponse={true}
/>

Voice-Controlled Search

const { transcript, speak } = useVoiceControl();

const handleSearch = async () => {
  const results = await searchAPI(transcript);
  speak(`Found ${results.length} results`);
};

Accessibility Features

<AudioRecorder onTranscript={setText} />
<VoiceButton onClick={() => speak(text)} />

Multilingual Apps

useSpeechRecognition({ language: 'es-ES' });
useSpeechSynthesis({ voice: 'af_bella' });

⚙️ Configuration

Next.js Integration

Add to next.config.js:

module.exports = {
  webpack: (config, { isServer }) => {
    if (!isServer) {
      config.resolve.fallback = {
        ...config.resolve.fallback,
        fs: false,
        path: false,
      };
    }
    return config;
  },
};

Web Worker Setup

STT uses Web Workers automatically. Ensure your build tool supports workers:

// Vite
import SpeechWorker from 'use-voice-control/speech/worker?worker';

// Webpack/Next.js
const SpeechWorker = require('use-voice-control/speech/worker.js');

🔐 Privacy & Security

• Client-side STT: Moonshine.js runs entirely in the browser—no audio leaves your device
• Optional Server TTS: Choose Kokoro (server-side) or Deepgram (with API key)
• No Tracking: No analytics or usage telemetry
• HTTPS Required: Microphone access requires secure context

🐛 Troubleshooting

Microphone Access Denied

• Ensure HTTPS or localhost
• Check browser permissions (Settings → Privacy → Microphone)
• Grant permission when prompted

Silent or Garbled Audio

• Adjust volume in useSpeechSynthesis options
• Try a different voice or provider
• Check speaker/headphone connection

STT Not Recognizing Speech

• Verify microphone is working (test in browser console)
• Speak clearly and at normal volume
• Check language setting matches spoken language

📦 Exports

// Hooks
export { useSpeechRecognition } from 'use-voice-control/hooks';
export { useSpeechSynthesis } from 'use-voice-control/hooks';
export { useVoiceControl } from 'use-voice-control/hooks';

// Components
export { AudioRecorder } from 'use-voice-control/components';
export { VoiceSelector } from 'use-voice-control/components';
export { AudioPlayer } from 'use-voice-control/components';

// Core API
export { generateSpeech } from 'use-voice-control/speech';
export type { TTSProvider, TTSOptions, KokoroVoice, DeepgramSpeaker } from 'use-voice-control/speech';

📄 License

rights.institute/PROSPER

🤝 Contributing

We welcome contributions! Please review CONTRIBUTING.md and open a PR.

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Discord: Join Community

🔗 Related Packages

• qwksearch-api-client - API client for search and content extraction
• agent-toolkit - Multi-provider LLM agent toolkit
• research-agent-ui - React UI components for research agents

📖 Documentation

• Voice Recognition with Moonshine.js
• Kokoro TTS Documentation
• Deepgram API Reference
• Web Audio API Guide

Please star this repo for updates! ⭐