@superapp_men/voice-recorder-capacitor

Voice recorder for SuperApp partner apps. Records audio through the SuperApp's native layer (Capacitor) with automatic checkpoint streaming — start once, receive audio chunks every second without interrupting the recording.

Install

npm install @superapp_men/voice-recorder-capacitor

How It Works

Partner App (iframe)          SuperApp (Capacitor host)
       |                              |
       |-- startRecording(config) --> |-- starts native mic
       |                              |-- starts AudioWorklet chunking
       |                              |
       | <-- checkpoint #1 (push) --- | (every 1s, automatic)
       | <-- checkpoint #2 (push) --- |
       | <-- checkpoint #3 (push) --- |
       |                              |
       |-- stopRecording -----------> |-- stops native mic
       | <-- final full recording --- |

• The native recording runs continuously — never stopped for checkpoints.
• An AudioWorklet captures raw PCM at the requested sample rate and pushes WAV chunks automatically.
• The partner app receives checkpoints via events — no polling, no manual calls.

Quick Start

1. Simple Recording (no checkpoints)

import { VoiceRecorder, AudioFormat, SampleRate } from "@superapp_men/voice-recorder-capacitor";

const recorder = new VoiceRecorder({ timeout: 10000, debug: true });

// Request permission
const permission = await recorder.requestPermission();
if (permission !== "granted") return;

// Record
await recorder.startRecording({
  audioConfig: {
    format: AudioFormat.WAV,
    sampleRate: SampleRate.SR_16000,
    bitDepth: 16,
    channels: 1,
  },
});

// ... user speaks ...

const result = await recorder.stopRecording();
// result.audioData  → base64-encoded WAV at 16kHz/16bit/mono
// result.duration   → total ms
// result.audioConfig → { format: "wav", sampleRate: 16000, bitDepth: 16, channels: 1 }

2. Recording with Auto Checkpoints

const recorder = new VoiceRecorder({ timeout: 10000, debug: true });

// Listen for checkpoints — they arrive automatically, no manual calls needed
recorder.on("checkpointCreated", ({ checkpoint }) => {
  console.log(`Chunk #${checkpoint.index}`, {
    segmentDuration: checkpoint.segmentDuration, // ~1000ms
    sampleRate: checkpoint.audioConfig.sampleRate, // 16000
    size: checkpoint.size,
  });
  // Send checkpoint.audioData to your STT API, save it, etc.
});

await recorder.requestPermission();

await recorder.startRecording({
  isCheckpoints: true,          // enable auto checkpoints
  checkpointInterval: 1000,     // one chunk every 1 second (min: 500ms)
  maxDuration: 120_000,         // max 2 minutes
  audioConfig: {
    format: AudioFormat.WAV,
    sampleRate: SampleRate.SR_16000,
    bitDepth: 16,
    channels: 1,
  },
});

// Checkpoints stream in automatically every 1s via the "checkpointCreated" event.
// No need to call createCheckpoint() — just listen.

// When done:
const result = await recorder.stopRecording();
// result.audioData       → full recording (from native plugin, converted to WAV)
// result.checkpoints     → array of all checkpoint objects
// result.checkpointCount → number of checkpoints

3. Auto-Stop on Silence

Stop the recording automatically once the user has spoken and then gone quiet — no need to call stopRecording() yourself.

const recorder = new VoiceRecorder({ timeout: 10000, debug: true });

// Auto-stop fires a "recordingStopped" event just like a manual stop, but
// with reason: "silence" so you can tell the two apart.
recorder.on("recordingStopped", ({ result, reason }) => {
  if (reason === "silence") {
    console.log("Stopped automatically — trailing silence detected");
  }
  // result.audioData / result.modelAiResult etc. are available either way
});

await recorder.requestPermission();

await recorder.startRecording({
  lang: "ar",
  useModelAi: true,
  referenceText: "...",
  autoStopOnSilence: true,   // opt-in, default false
  silenceThreshold: 0.02,    // optional — linear RMS 0-1, default 0.02
  silenceDurationMs: 1500,   // optional — ms of trailing silence before stopping, default 1000
  minSpeechDurationMs: 300,  // optional — speech required before silence can trigger a stop, default 300
});

// Nothing else to do — either the user's own stopRecording() call resolves
// normally, OR the "recordingStopped" event fires on its own with reason:
// "silence". Do NOT call stopRecording() after that — the session is already
// closed on the SuperApp side and it will reject with "Not currently recording".

Tuning notes:

• silenceThreshold depends on the device's mic sensitivity and ambient noise — the default is a starting point, not a guarantee. If silence never triggers, lower it; if it triggers on background noise, raise it.
• minSpeechDurationMs exists so mic warm-up / the user positioning their phone before speaking isn't mistaken for "done talking" — keep it non-zero.
• Works in both plain and isCheckpoints: true mode.

4. React Example

import { useEffect, useState } from "react";
import {
  VoiceRecorder,
  RecorderState,
  AudioFormat,
  SampleRate,
  formatDuration,
  type RecordingResult,
  type Checkpoint,
} from "@superapp_men/voice-recorder-capacitor";

function AudioRecorder() {
  const [recorder] = useState(() => new VoiceRecorder({ timeout: 10000, debug: true }));
  const [state, setState] = useState(RecorderState.IDLE);
  const [duration, setDuration] = useState(0);
  const [checkpoints, setCheckpoints] = useState<Checkpoint[]>([]);
  const [recording, setRecording] = useState<RecordingResult | null>(null);
  const [error, setError] = useState<string | null>(null);

  useEffect(() => {
    const unsubs = [
      recorder.on("stateChange", ({ state }) => setState(state)),
      recorder.on("progress", ({ duration }) => setDuration(duration)),
      recorder.on("checkpointCreated", ({ checkpoint }) => {
        setCheckpoints((prev) => [...prev, checkpoint]);
      }),
      recorder.on("error", ({ message }) => setError(message)),
    ];
    return () => { unsubs.forEach((u) => u()); recorder.destroy(); };
  }, [recorder]);

  const isRecording = state === RecorderState.RECORDING;

  const start = async () => {
    setError(null);
    setRecording(null);
    setCheckpoints([]);
    const p = await recorder.requestPermission();
    if (p !== "granted") { setError("Permission denied"); return; }

    await recorder.startRecording({
      isCheckpoints: true,
      checkpointInterval: 1000,
      audioConfig: {
        format: AudioFormat.WAV,
        sampleRate: SampleRate.SR_16000,
        bitDepth: 16,
        channels: 1,
      },
    });
  };

  const stop = async () => {
    const result = await recorder.stopRecording();
    setRecording(result);
  };

  return (
    <div>
      <p>State: {state} | Duration: {formatDuration(duration)}</p>
      {error && <p style={{ color: "red" }}>{error}</p>}

      <button onClick={start} disabled={isRecording}>Start</button>
      <button onClick={stop} disabled={!isRecording}>Stop</button>

      {/* Live checkpoints */}
      {checkpoints.map((cp) => (
        <div key={cp.id}>
          <p>Checkpoint #{cp.index} — {cp.audioConfig.sampleRate}Hz, {cp.segmentDuration}ms</p>
          <audio controls src={`data:audio/wav;base64,${cp.audioData}`} />
        </div>
      ))}

      {/* Final result */}
      {recording && (
        <div>
          <p>Done — {formatDuration(recording.duration)}, {recording.checkpointCount} checkpoints</p>
          <audio controls src={`data:audio/wav;base64,${recording.audioData}`} />
        </div>
      )}
    </div>
  );
}

API

new VoiceRecorder(config?)

| Option | Type | Default | Description | | --------- | --------- | ------- | ------------------------ | | timeout | number | 5000 | Request timeout (ms) | | debug | boolean | false | Enable console logging |

Methods

| Method | Returns | Description | | --------------------- | -------------------------- | ---------------------------------------------- | | isAvailable() | Promise<boolean> | Can the device record? | | checkPermission() | Promise<PermissionStatus> | Current mic permission status | | requestPermission() | Promise<PermissionStatus> | Ask the user for mic access | | startRecording(config?) | Promise<void> | Start recording with optional config | | stopRecording() | Promise<RecordingResult> | Stop and get the full result | | createCheckpoint() | Promise<Checkpoint> | On-demand checkpoint (rarely needed with auto mode) | | getStatus() | Promise<RecordingStatus> | Current status from the SuperApp | | getState() | RecorderState | Local state | | isCurrentlyRecording() | boolean | Quick check | | getDuration() | number | Elapsed ms since start | | getCheckpoints() | Checkpoint[] | All checkpoints so far | | on(event, callback) | () => void | Subscribe (returns unsubscribe fn) | | off(event, callback) | void | Unsubscribe | | destroy() | void | Cleanup everything |

startRecording(config)

| Option | Type | Default | Description | | ------------------- | ------------- | -------- | ---------------------------------------------- | | isCheckpoints | boolean | false | Enable auto checkpoint streaming | | checkpointInterval| number | 1000 | Ms between checkpoints (min 500) | | useModelAi | boolean | false | Enable/disable AI model processing | | lang | 'fr' \| 'ar' \| 'math' | — | Language model: 'fr' for French, 'ar' for Arabic, 'math' for spoken-number recognition | | referenceText | string | — | Expected/correct text for this recording session (e.g. for pronunciation assessment) | | maxDuration | number | — | Auto-stop after this many ms | | autoStopOnSilence | boolean | false | Auto-stop once trailing silence is detected after speech — see Auto-Stop on Silence | | silenceThreshold | number | 0.02 | Linear RMS (0-1) below which audio counts as silence. Only used when autoStopOnSilence is true | | silenceDurationMs | number | 1000 | Ms of trailing silence required before auto-stopping. Only used when autoStopOnSilence is true | | minSpeechDurationMs | number | 300 | Ms of speech required before silence can trigger a stop. Only used when autoStopOnSilence is true | | timeout | number | 5000 | Request timeout (ms) | | metadata | Record<string, unknown> | — | Custom metadata | | audioConfig.format| AudioFormat | WAV | AudioFormat.WAV or AudioFormat.PCM | | audioConfig.sampleRate | number | 16000 | Hz (8000–48000) | | audioConfig.bitDepth | number | 16 | 8, 16, 24, or 32 | | audioConfig.channels | number | 1 | 1 (mono) or 2 (stereo) |

Events

| Event | Payload | Description | | ------------------- | ------------------------------------ | ---------------------------------- | | stateChange | { state, previousState } | Recorder state changed | | recordingStarted | { sessionId, config } | Recording began | | recordingStopped | { result: RecordingResult, reason?: 'silence' } | Recording ended — reason: 'silence' means autoStopOnSilence triggered it, absent means stopRecording() was called | | checkpointCreated | { checkpoint: Checkpoint } | A chunk arrived (auto or manual) | | progress | { duration, checkpointCount } | Fires every 500ms while recording | | error | { code, message } | Something went wrong |

RecordingResult

{
  audioData: string;            // base64 WAV of the full recording
  duration: number;             // total ms
  size: number;                 // bytes
  timestamp: number;            // when recording finished
  audioConfig: {                // actual output config
    format: string;
    sampleRate: number;
    bitDepth: number;
    channels: number;
  };
  checkpointCount: number;
  checkpoints?: Checkpoint[];   // present if isCheckpoints was true
  modelAiResult?: Audio2PhonemeFr | Audio2PhonemeAr | Audio2PhonemeMath; // AI model output (if enabled) — shape depends on `lang`
}

Checkpoint

{
  id: string;                   // unique ID
  index: number;                // 1-based
  audioData: string;            // base64 WAV of this segment
  duration: number;             // elapsed ms from recording start
  segmentDuration: number;      // duration of this chunk in ms
  size: number;                 // bytes
  timestamp: number;
  audioConfig: {                // matches requested config
    format: string;
    sampleRate: number;
    bitDepth: number;
    channels: number;
  };
  modelAiResult?: Audio2PhonemeFr | Audio2PhonemeAr; // AI model output for this checkpoint (if enabled) — shape depends on `lang`
}

Audio2PhonemeFr / Audio2PhonemeAr / Audio2PhonemeMath

modelAiResult on Checkpoint and RecordingResult is a union of three shapes, selected by lang in startRecording(config):

• lang: 'fr' → Audio2PhonemeFr (audio-to-phoneme transcription)
• lang: 'ar' → Audio2PhonemeAr (pronunciation assessment)
• lang: 'math' → Audio2PhonemeMath (spoken-number recognition)

// Audio2PhonemeFr
{
  transcript: string;       // full IPA string
  confidence: number;       // overall confidence (0–1)
  latencyMs: number;
  language: string;         // e.g. "fr-FR"
  modelVersion: string;     // e.g. "v1"
  audio: {
    durationMs: number;
    sampleRateHz: number;
    channels: number;
    encoding: string;       // currently "pcm16"
  };
  timestamps?: {            // per-phoneme timeline (optional)
    phoneme: string;        // single IPA symbol, e.g. "œ̃", "ʃ", "a"
    startMs: number;
    endMs: number;
    confidence: number;     // 0–1
    candidates: {           // alternative phoneme guesses
      phoneme: string;
      probability: number;
    }[];
  }[];
}

// Audio2PhonemeAr
{
  accepted: boolean;
  label: string;
  scorePercent: number;
  assessmentVersion: string;
  words: {
    id: string;
    referenceWord: string;
    producedText: string;
    status: "correct" | "mispronounced" | string;
    faults: string[];
  }[];
  faults: string[];
  debug: {
    referencePhonemes: string;
    detectedPhonemes: string;
    rawEspeakIpa: string;
    detectedText: string;
    topN: {
      applied: boolean;
      reason: string | null;
      n: number;
      threshold: number;
    };
  };
}

// Audio2PhonemeMath
{
  expected: {
    number: number;
    score: number;
    rank: number;
    isMostLikely: boolean;
    relativeToBestScore: number;
    isPresent: boolean;
  };
  hasDetection: boolean;
  detectedTokens: string[];        // raw phoneme/token sequence detected from the audio
  mostLikely: {
    number: number;
    score: number;
    rank: number;
    distance: number;
    errorRate: number;
  };
  likelyNumbers: {                 // ranked candidates, best first
    number: number;
    score: number;
    rank: number;
    distance: number;
    errorRate: number;
  }[];
}

Example:

{
  "expected": {
    "number": 1,
    "score": 0.96,
    "rank": 2,
    "isMostLikely": false,
    "relativeToBestScore": 0.97,
    "isPresent": true
  },
  "hasDetection": true,
  "detectedTokens": ["w", "aa", "H", "i", "d"],
  "mostLikely": {
    "number": 41,
    "score": 0.99,
    "rank": 1,
    "distance": 0.1,
    "errorRate": 0.01
  },
  "likelyNumbers": [
    { "number": 41, "score": 0.99, "rank": 1, "distance": 0.1, "errorRate": 0.01 },
    { "number": 1, "score": 0.96, "rank": 2, "distance": 0.2, "errorRate": 0.04 },
    { "number": 40, "score": 0.93, "rank": 3, "distance": 0.5, "errorRate": 0.07 }
  ]
}

Enums

enum RecorderState {
  IDLE, REQUESTING_PERMISSION, READY, RECORDING, PAUSED, STOPPED, ERROR
}

enum AudioFormat { WAV = "wav", PCM = "pcm" }

enum SampleRate {
  SR_8000 = 8000, SR_11025 = 11025, SR_16000 = 16000,
  SR_22050 = 22050, SR_44100 = 44100, SR_48000 = 48000
}

type PermissionStatus = "granted" | "denied" | "prompt" | "unknown";

Recommended Configs

Voice / STT (low bandwidth):

{ format: AudioFormat.WAV, sampleRate: 16000, bitDepth: 16, channels: 1 }

High quality:

{ format: AudioFormat.WAV, sampleRate: 44100, bitDepth: 16, channels: 2 }

For SuperApp Developers

If you're building the SuperApp side that handles these messages, import from the /superapp entry point:

import {
  MessageType,
  RecorderState,
  type SuperAppMessage,
  type StartRecordingPayload,
  type RecordingResult,
  type Checkpoint,
} from "@superapp_men/voice-recorder-capacitor/superapp";

The SuperApp must:

1. Listen for MessageType.START_RECORDING via postMessage
2. Start the native Capacitor recording + an AudioWorklet for chunking
3. Push MessageType.CHECKPOINT_PUSH messages to the iframe automatically
4. On MessageType.STOP_RECORDING, stop everything and return the full RecordingResult

See VoiceRecorderPackageService.ts for the reference implementation.

Troubleshooting

"Already recording" errors — The old start/stop checkpoint approach caused this. v1.1.0 uses continuous recording with AudioWorklet chunking — this error should no longer occur.

Checkpoints not arriving — Make sure isCheckpoints: true is in the config. Listen to "checkpointCreated" events — checkpoints are pushed automatically.

Wrong sample rate in checkpoints — The SuperApp creates an AudioContext at the requested rate. If the device can't honour it, a lightweight resample is applied. Check checkpoint.audioConfig.sampleRate to verify.

stopRecording() throws "Not currently recording" after silence auto-stop — Once autoStopOnSilence triggers, the SuperApp has already closed the session. Don't call stopRecording() afterward — listen for the recordingStopped event (with reason: 'silence') instead of calling stopRecording() yourself.

Debug mode — Pass debug: true to see all bridge messages:

const recorder = new VoiceRecorder({ debug: true });

License

MIT

Support

• Email: [email protected]