@memoreco/memoreco-js

The official Memoreco Web SDK for JavaScript and TypeScript. Record, upload, and manage media assets with optional AI processing.

Features

• 🎥 Screen & Camera Recording - Built on top of @memoreco/recorder
• 📤 Smart Upload Strategy - Automatic selection between Memoreco-managed and custom storage
• 🤖 AI Processing - Integrate AI services for transcription, analysis, and enhancement
• 🎬 Media Player - Embed and play recordings with customizable themes
• 📊 Real-time Events - Listen to recording, upload, and AI processing events
• 🔧 TypeScript Support - Full type definitions included
• ⚛️ React Integration - Hooks and components for React apps
• 🎯 Smart UI Behavior - Device overlay auto-hides during capture/recording and restores when finished
• 🧠 AI Overlay - Optional guidance overlay during recording (not captured in screen recordings)
• 🎛️ Recording Controls - Simplified API for managing recording UI state
• 🖼️ Recording Gallery - Automatic display of captured videos and photos below preview

Recent Updates

• 2026-01-19: v0.4.3 - Bundle @memoreco/shared at build time so external devs don't need to install it separately.
• 2026-01-19: v0.4.2 - Relax @memoreco/recorder peer dependency to include ^0.3.0 (pairs with recorder v0.3.0).
• 2026-01-19: v0.4.1 - Fix npm installation by replacing the internal workspace:* dependency on @memoreco/shared with a published semver range.
• 2026-01-19: v0.4.0 - AI Assist improvements (start decision logic, countdown handling), MemoryModule integration, AI guidance use cases with gaze tracking, and streaming transcription AI assist config.
• 2026-01-16: AI Assist Persona IDs - SDK AI Assist now requires personaId and no longer sends systemPrompt or role in recording configs.
• 2025-12-06: UI Consistency & Polish (v0.3.19) - Major visual refinement pass to match Figma designs:
  • Countdown Timer: Fixed positioning (36px/28px from edges), font (Neue Haas Medium 600), size (16px red text), dot (16x16px), and pill dimensions (89x40px).
  • AI Overlay: Eliminated duplicate borders (CSS + monoco library conflict), fixed border sync during fade-in with requestAnimationFrame, prevented premature dismissal during typewriter animation.
  • Streaming Transcript: Enforced Inter font with !important, centered pills that expand left when reaching max width, fixed padding (container 16px, pill 16px/32px), eliminated background opacity changes.
  • Review & Pause Buttons: Standardized all buttons to pill style (32px height, 50px border-radius, Neue Haas Medium).
  • All UI components now load correct fonts from CDN (Neue Haas Display Roman/Medium, Inter).
• 2025-12-02: Language-Aware Captions & Provider Sync - Real-time captions now follow Netflix-style guidelines (max two lines, ~42 characters each) so long sentences wrap into stacked subtitle lines instead of stretching across the frame. We also keep streaming provider preferences in lockstep with the recorder config, ensuring Speechmatics-only builds stay on Speechmatics all the way down to the transcription service.
• 2025-12-02: AI Overlay Countdown & SSE Resilience - The AI Assist overlay now shows the full countdown duration immediately, even while the typewriter animation is running, so interviewees see exactly how much time remains before the prompt retires. On the transport side, the SSE client now reuses the centralized API constants (no more hard-coded /v1) and automatically retries with capped backoff whenever a browser drops the connection, preventing the “three interventions then silence” behaviour during longer takes.
• 2025-11-30: AI Assist Preview Integration - When ai_assist is included in aiServices, the SDK now auto-connects to the AI Assist SSE stream, renders the translucent overlay inside the camera container, and tears everything down as soon as the recording stops or is cancelled. No extra wiring is needed in apps—the helper mirrors the persona configured on the recording and keeps overlays hidden from the captured media.
• 2025-11-29: QR Handoff Complete - Release 0.3.0 marks the first stable QR handoff feature. Desktop users can open a QR overlay, scan with their phone, and the captured photo/video streams back to the desktop in real-time via SSE. Includes fetch-based SSE (bypasses ngrok interstitial), mirror-aware transforms, credit-check whitelisting, and proper submission sequencing so the desktop overlay always receives the submission_complete event.
• 2025-11-29: QR Handoff Mobile Fixes - Release 0.2.39 fixes logo and sounds not loading on mobile /r/ recording pages by whitelisting /brand/ and /sounds/ paths in waitlist middleware. Fixes recording creation failing with 500 error by using the request UUID (not slug) in ephemeral token metadata. Restores mirror transform after QR overlay closes so selfie preview stays flipped.
• 2025-11-29: QR Handoff Production Fixes - Release 0.2.38 fixes 403 errors on public recording request pages by whitelisting the endpoint in visibility middleware. Fixes 401 errors when cancelling QR handoff sessions by using the dedicated cancel-handoff endpoint with proper ephemeral token auth. Also suppresses PTZ OverconstrainedError warnings for devices that don't support zoom/pan/tilt, and fixes token provider for voice demo in profile photo explainer.
• 2025-11-28: QR Handoff SSE Improvements - Release 0.2.37 replaces native EventSource with fetch-based SSE implementation for QR handoff, adding support for the ngrok-skip-browser-warning header to bypass ngrok free tier interstitial pages during development. Also adds Cache-Control to allowed CORS headers.
• 2025-11-25: Staged Photo Validation Loading - New preloadOnInit: 'staged' option loads face detection on SDK init (~2-3s) and auto-loads NSFW detection when camera starts. This improves page load performance while keeping capture responsive. Also adds preloadFaceDetection(), preloadNsfwDetection(), isFaceDetectionReady(), and isNsfwDetectionReady() methods for manual control. See Photo Validation docs.
• 2025-11-24: Release 0.2.19 adds URL Thumbnail Capture for Voice Mode - automatically detects URLs in user input, captures full-page screenshots, displays loading states, and shows clickable previews. Includes new useThumbnailCapture and useDebouncedUrlParser hooks, plus ThumbnailCard, ThumbnailPreview, and ThumbnailStrip UI components in @memoreco/ui. Thumbnails continue loading in chat after messages are sent.
• 2025-11-20: Release 0.2.15 adds voiceMode.mountVoiceButton(container, options) helper for instant-loading Voice Mode with eager resource preloading (AudioWorklet, intro sound) before initialization. External devs no longer need to manually wire up promises, handlers, and DOM mounting—just call the helper and it handles initialization, styling, and callbacks automatically.
• 2025-11-20: Release 0.2.14 adds clearStreamingTranscript() API for chat interfaces to reset the buffer mid-recording, hover stop icon (white square appears on hover during recording), and intro sound cooldown (prevents duplicate playback). Also fixes loader rendering to preserve button structure.
• 2025-11-19: Release 0.2.13 fixes audio level indicator (pulsating mask inside microphone) scaling by converting AnimatedMicrophone SVGs to use viewBox and percentage dimensions, ensuring proper resizing with the button.
• 2025-11-19: Release 0.2.12 fixes preview overlay visibility (now fully hidden with display: none when showPreviewOverlay: false) and progress ring scaling (SVG now uses viewBox and percentage dimensions for proper button resizing).
• 2025-11-19: Release 0.2.11 adds showStatusBubble and showPreviewOverlay toggles to Voice Mode so inline chat integrations can hide SDK-managed UI elements while streaming transcripts into their own inputs. Also bumps the exposed SDK_VERSION constant so provider logs report 0.2.11.
• 2025-11-19: Release 0.2.10 fixes warm-mic double intro sound—Voice Mode reactivations now play a single chime by gating the preview layer's audio when the mic is already warm. Adds playIntroSound option to CameraPreviewOptions for fine-grained control. Includes SDK_VERSION constant update.
• 2025-11-19: Release 0.2.9 (deprecated - use 0.2.10) - initial fix for warm-mic double intro sound.
• 2025-11-19: Release 0.2.8 streamlines Voice Mode for chat integrations: the Siri-style button now supports click-to-talk activation, the intro sound only plays once per cold start, the streaming transcript overlay gets balanced padding plus text-replacement hooks, and prompt enhancement updates both the preview and chat input as soon as recordings stop.
• 2025-11-12: Release 0.2.7 fixes two critical Firefox issues: (1) voice status bubble positioning now uses position: fixed matching the button's strategy for correct resize behavior, and (2) resolves "Connecting AudioNodes with different sample-rate" error by creating fresh AudioContexts at the mic's native rate (e.g., 48kHz) with worklet-based downsampling. The SDK now reports its own version plus the recorder and UI package versions via sdk.version (object) as well as convenience fields sdk.sdkVersion, sdk.recorderVersion, and sdk.uiVersion.
• 2025-11-12: Release 0.2.6 keeps the voice status bubble pinned to the button during viewport changes, disables verbose logging in production by default, and pairs with recorder 0.2.5 for Firefox-safe audio fallbacks.
• 2025-11-11: Release 0.2.5 regenerates the internal API client so webhook testing helpers stay available during development and syncs recorder dependency requirements.
• 2025-11-11: Release 0.2.4 hardens push-to-talk race handling—recordings always stop on keyup even if the previous command was still “Understanding…”, preventing phantom sessions during rapid retries.
• 2025-11-10: Release 0.2.3 refreshes the Siri for Sites MCP quick start, updates voice-task samples, and syncs docs ahead of the npm publish.
• 2025-11-11: Voice Tasks debug logging now respects production mode so live deployments stay clean while errors remain visible.
• 2025-11-10: Warm mic pre-roll capture, streaming transcript stabilization, and token lifecycle fixes ensure instant Speechmatics sessions with accurate UI updates.
• 2025-10-30: Recording flows fall back to the permission helper when preview streams are missing, ensuring users see the guided modal before recording fails. New SDK config flags (usePermissionHelper, permissionHelperOptions) let you customize this per recording.
• 2025-10-29: Voice Tasks now triggers the permission helper before the first recording so users see the browser prompt instead of a silent failure. Note: If permissions are already blocked in browser settings, the helper is skipped and a standard error is shown.
• 2025-10-29: Added idle overlay diagnostics in VoiceTasksModule and DeviceOverlay to trace audio idle screen reappearance.

Installation

npm install @memoreco/memoreco-js

Quick Start

import { createMemoreco } from "@memoreco/memoreco-js";

const memoreco = await createMemoreco({
  apiKey: "your-api-key",
  debug: true,
});

const logger = memoreco.getLogger("quick-start");

// Check browser capabilities
const capabilities = memoreco.getCapabilities();
logger.info("Screen recording supported", { supported: capabilities.screenRecording });

// Listen to events with structured logging
memoreco.on("recording.start", (event) => {
  logger.info("Recording started", { recordingId: event.recordingId });
});

memoreco.on("upload.complete", (event) => {
  logger.info("Upload complete", { assetUrl: event.url });
});

memoreco.on("upload.cancelled", (event) => {
  logger.warn("Upload cancelled", { reason: event.reason });
});

Cancel uploads

// Abort the active upload by recordingId
memoreco.recordings.cancelUpload(recordingId);

Configuration

interface MemorecoConfig {
  /** API key for authentication */
  apiKey: string;
  /** API base URL (optional, defaults to production) */
  apiUrl?: string;
  /** 
   * Debug mode for additional logging
   * When enabled, logs API requests, responses, events, and internal operations to console
   * Includes timing information, error details, and state changes
   * Should be disabled in production for performance and security
   */
  debug?: boolean;
  /** 
   * Custom user agent identifier for API requests
   * Helps identify your application in logs and analytics
   * Format: "YourApp/1.0.0 ([email protected])"
   * If not provided, uses default SDK user agent
   */
  userAgent?: string;
  /** UI behavior configuration */
  ui?: {
    /** AI overlay configuration */
    aiOverlay?: {
      /** Enable/disable the AI overlay (default: false) */
      enabled?: boolean;
      /** Custom text to display in the overlay */
      defaultMessage?: string;
      /** Position of the overlay */
      position?: 'top-left' | 'top-right' | 'bottom-left' | 'bottom-right' | 'center';
      /** Custom CSS styles for the overlay container */
      style?: Partial<CSSStyleDeclaration>;
      /** Whether to show a spinner in the overlay (default: true) */
      showSpinner?: boolean;
      /** Whether to show a timer in the overlay (default: true) */
      showTimer?: boolean;
    };
    /** 
     * Enable review mode for captures (default: false)
     * When enabled, users can preview and accept/reject media before it's uploaded and persisted.
     * Applies to all capture types: photos, videos, audio, and screen recordings.
     */
    reviewMode?: boolean;
  };
}

UI Behavior

The SDK automatically manages UI elements during capture and recording:

• Device Overlay: Automatically slides out during photo capture, multi-shot sequences, and video recording
• AI Overlay: Optional guidance overlay during recording (configurable via ui.aiOverlay)
• Gallery Display: Captured videos and photos automatically appear below the camera preview
• Overlay Restoration: Device overlay slides back in when capture/recording finishes and no blocking overlays (QR, pause, restart) are active

Permission Helper (First-Time Recording)

When you enable the built-in permission helper, the SDK displays Memoreco's guided modal before calling getUserMedia. This ensures the browser prompt appears on the first click instead of failing silently when access is missing.

await memoreco.camera.showPreview("voice-tasks-preview", {
  selectedModeId: "audio",
  usePermissionHelper: true,
  permissionHelperOptions: {
    explanation: "Allow microphone access so Memoreco can capture your voice command.",
    darkMode: true
  }
});

Voice Tasks integrations now pass usePermissionHelper automatically, so the helper runs whenever a user hasn't granted microphone or camera permissions yet.

Review Mode

Full gallery documentation

Enable review mode to give users the ability to preview and accept/reject captures before they're uploaded:

const sdk = new Memoreco({
  apiKey: 'your-api-key',
  ui: {
    reviewMode: true  // Enable review mode for all captures
  }
});

// Listen for review events
const reviewLogger = sdk.getLogger('review-mode');
sdk.on('snapshot.taken', (event) => {
  reviewLogger.info('Capture ready for review', { snapshotId: event.snapshotId });
});

sdk.on('snapshot.accepted', (event) => {
  reviewLogger.info('User accepted capture', { assetId: event.assetId, assetUrl: event.url });
});

sdk.on('snapshot.rejected', (event) => {
  reviewLogger.warn('User rejected capture', { snapshotId: event.snapshotId });
});

// Capture with review
const result = await sdk.capture({ media: 'photo', source: 'camera' });
// Review overlay will appear automatically
// User can click "Accept" to upload or "Retake" to discard

Review Mode Behavior:

• Works for all capture types: photos, videos, audio, and screen recordings
• Shows an overlay with the captured media and Accept/Retake buttons
• Upload is deferred until user accepts the capture
• No server-side storage or asset creation until acceptance
• Rejected captures are immediately discarded from memory
• Events: snapshot.taken (pending), snapshot.accepted (uploaded), snapshot.rejected (discarded)

Streaming Transcript Overlay

Need to see what the AI Assist service receives in real time? Enable the optional streaming transcript overlay so the live transcript floats over the preview:

const sdk = new Memoreco({
  apiKey: 'your-api-key',
  ui: {
    streamingTranscriptOverlay: {
      enabled: true // Default: false
    }
  }
});

When enabled, the SDK mounts the StreamingTranscript component inside the camera container so every transcript event is visible. Keep it disabled in production if you prefer a cleaner recording UI.

Recording Controls

Full documentation

The Recording Controls API simplifies recording UI management by providing a single state subscription instead of manual event wiring:

// Get the recording controls manager
const controls = sdk.getRecordingControls();
const controlsLogger = sdk.getLogger('recording-controls');

// Subscribe to state changes
controls.onStateChange((state) => {
  // state is: 'idle' | 'recording' | 'paused' | 'error'
  controlsLogger.info('Recording state updated', { state });
  
  // Update your UI based on state
  updateRecordingButtons(state);
});

// Helper methods
if (controls.isRecording()) {
  controlsLogger.info('Currently recording');
}

Benefits:

• ✅ No need to manually wire up multiple event listeners
• ✅ SDK automatically manages AI overlay mounting/unmounting
• ✅ Single source of truth for recording state
• ✅ Type-safe state transitions
• ✅ Simplified integration for UI developers

Browser Compatibility

The SDK automatically detects browser capabilities:

• Screen Recording: Chrome 72+, Firefox 66+, Safari 13+
• Camera Recording: Chrome 53+, Firefox 36+, Safari 11+
• Audio Recording: Chrome 47+, Firefox 29+, Safari 11+
• File Upload: All modern browsers

Architecture

The SDK is built with a service-oriented architecture for maximum maintainability and performance:

@memoreco/memoreco-js
├── Core SDK (Memoreco class - 3,050 lines, reduced from 5,500)
├── Event System (EventEmitter)
├── Core Services:
│   ├── AuthManager (authentication & tokens)
│   ├── NetworkManager (connectivity management)
│   └── SystemLockHandler (system events)
├── Preview Services:
│   ├── PreviewController (camera lifecycle)
│   ├── InactivityManager (timeout handling)
│   ├── DeviceOverlay (device selection UI)
│   ├── Overlays (UI overlays & modals)
│   └── SnapshotEffects (visual/audio effects)
├── Capture & Display:
│   ├── CaptureCoordinator (capture orchestration)
│   └── MediaDisplay (media rendering)
├── Utilities:
│   ├── Logger (centralized logging)
│   ├── DOM utilities (element manipulation)
│   └── UI constants (theming & styles)
└── Types & Tests (TypeScript definitions & 100+ unit tests)

Key Improvements:

• 🔧 45% code reduction (5,500 → 3,050 lines) with 100% API compatibility
• 🧪 100+ unit tests with comprehensive service coverage
• 🎯 14 focused services with single responsibilities
• ⚡ Better performance through optimized service architecture
• 📚 Enhanced maintainability with clear separation of concerns

For detailed architecture documentation, see the repo guide: SDK_FILE_STRUCTURE.md.

Dependencies

• @memoreco/recorder - Screen and camera recording
• @memoreco/api-sdk-public - API client
• @memoreco/types - Shared types
• @memoreco/utils - Utility functions

Release Workflow

Follow the internal release checklist: docs/PUBLISHING.md for pnpm run release:check, the automated scripts/publish.mjs helper, and post-publish tagging guidance.

Support

For questions and support:

• 📧 Email: [email protected]
• 📚 Documentation: docs.memoreco.com