Resound

Audio production for video. Track-based, markdown-driven, Remotion-ready.

Requires an ElevenLabs API key. Set ELEVENLABS_API_KEY in your environment or add api_key to your config file.

What is Resound?

Resound is a CLI tool that generates audio for video projects. You define tracks in markdown files, and Resound generates audio files with timing data that integrates seamlessly with Remotion.

Key features:

• Track-based workflow — Separate files for dialog, sound effects, and music
• Markdown-driven — Write dialog naturally, define sounds with prompts
• Incremental generation — Only regenerate what changed
• Remotion integration — Outputs manifest.json with markers and timing
• Self-documenting — Run resound for full help, resound --help <topic> for details

Quick Start

# In a Remotion project (auto-detects and configures)
npx create-video@latest my-video
cd my-video
npm install @sanity-labs/resound
npx resound init      # Creates config, tracks/, and src/audio/
npx resound generate  # Generate audio

resound init auto-detects Remotion projects and scaffolds everything:

• resound.config.yaml with output to public/audio/
• src/audio/ with pre-loaded manifest and ready-to-use components

Project Structure

my-video/
├── resound.config.yaml       # Project config
├── tracks/
│   ├── 01-dialog.md          # Multi-speaker dialog
│   ├── 02-sfx.md             # Sound effects
│   └── 03-music.md           # Background music
└── public/audio/             # Generated output
    ├── manifest.json         # Timing + markers for Remotion
    ├── dialog.mp3
    ├── dialog.timing.json    # Word-level timestamps
    ├── sfx.mp3
    └── music.mp3

Track Types

Dialog

Multi-speaker conversation with natural turn-taking:

---
type: dialog
voices:
  sarah: EXAVITQu4vr4xnSDxMaL
  bob: iP95p4xoKVk53GoZ742B
---

@sarah: Welcome to the demo! {#intro}

@bob: [excited] This looks amazing!

<!-- pause: 0:00.5 -->

@sarah: Let me show you how it works. {#demo-start}

Sound Effects

Define sounds once, place them by name:

---
type: sfx
sounds:
  # Simple: just a prompt
  whoosh: "smooth digital transition sound"
  pop: "soft notification pop"
  
  # With duration: for ambiance, typing, loops
  typing:
    prompt: "keyboard typing, mechanical keys"
    duration: 5.0
---

0:00           whoosh
intro+0.5      pop
demo-begins    typing    # 5 seconds of typing
demo-start     whoosh

Music

Background music with fades and ducking:

---
type: music
volume: 0.3
fade_in: 0:02
fade_out: 0:03
duck_to: 0.15
---

Gentle ambient electronic, soft pads, optimistic mood.

Remotion Integration

resound init scaffolds src/audio/ with everything pre-configured:

// Simple usage - everything pre-loaded
import { Audio, manifest, useTiming } from './audio';
import { loadTiming } from 'resound/remotion';
import dialogTimingJson from '../public/audio/dialog.timing.json';

const dialogTiming = loadTiming(dialogTimingJson);

export const MyVideo = () => {
  const { currentLine, visibleLines } = useTiming(dialogTiming);
  
  return (
    <>
      <Audio />
      {/* Your visuals synced to currentLine */}
    </>
  );
};

Available Hooks

| Hook | Description | |------|-------------| | useMarker(manifest, name) | Get a marker's frame number | | useMarkers(manifest) | Get all markers as { name: frame } | | useTiming(timing) | Word-level timing for streaming text effects | | useDucking(manifest, options) | Calculate volume with auto-ducking | | useIsDucking(manifest) | Check if currently in a ducking range |

Type-Safe Loaders

import { loadManifest, loadTiming } from 'resound/remotion';
import manifestJson from '../public/audio/manifest.json';
import dialogTimingJson from '../public/audio/dialog.timing.json';

const manifest = loadManifest(manifestJson);
const timing = loadTiming(dialogTimingJson);

Commands

resound                       # Show help
resound --help <topic>        # Detailed help (dialog, sfx, music, markers, remotion)

resound init                  # Create new project
resound new <type> <name>     # Create track from template
resound status                # Show project state
resound generate              # Generate audio (incremental)
resound generate --force      # Regenerate everything
resound markers               # List all markers
resound voices                # List available voices

Configuration

# resound.config.yaml
name: my-project
fps: 30
output: public/audio

voices:
  sarah: EXAVITQu4vr4xnSDxMaL
  bob: iP95p4xoKVk53GoZ742B

# Or use ELEVENLABS_API_KEY env var
# api_key: sk_...

Requirements

• Node.js 18+
• ElevenLabs API key

License

MIT