Vidfarm

Vidfarm is a serverless composition studio for short-form video. Directors fork templates, edit compositions on a timeline in the Trackpad Editor, render MP4s with the native HyperFrames renderer (cloud Lambda fan-out, or free in-process on a local vidfarm serve box), and share results.

Everything runs on AWS Lambda + DynamoDB + S3. There is no long-lived worker. It scales to zero at idle.

Where to start

• Using Vidfarm as a director → SKILL.director.md Login, forking templates, editing in the Trackpad Editor, auto-decompose, publishing, sharing, billing.

• Understanding how Vidfarm works internally → SKILL.platform.md Architecture, data model, composition runtime, publish pipeline, deployment, migrations.

Repository layout

• src/ — API server (Hono, deployed as container Lambda)
• demo/ — Trackpad Editor React app (deployed to CloudFront under /editor/)
• infra/cdk/ — CDK stacks for staging and production
• infra/lambda/ — Lambda entry points (api, editor-chat, primitive-media, job-runner, billing, ready-post-zip, job-sweeper)
• templates/ — Draft composition templates checked into the repo
• scripts/ — One-shot ops and migration scripts
• auto-create-hyperframe-templates/ — Extractor prompts / schemas for generating new templates from source footage

Deploy

# staging first, always
npm run cdk:synth:staging-serverless
npm run cdk:deploy:staging-serverless

# then production
npm run cdk:deploy:prod-serverless

Never call bare cdk deploy — the npm scripts load the correct .env.<stage> and select the right stack.

Local editor: vidfarm serve

Boot the full Vidfarm editor locally — single origin, disk-backed records + storage — so an AI agent (Claude Code, Codex, etc.) edits composition files on disk while a human finishes in the browser, both sharing one source of truth with live sync. Render runs in-process on your box, free (no cloud, no wallet charge).

npx -y @mevdragon/vidfarm-devcli serve
# → boots the full backend on http://localhost:3000 (records + storage on disk)
# → auto-provisions a local user and opens the editor, pre-authed
# → agents edit files under ./.vidfarm-local/storage — the browser live-morphs on save

Everything runs locally (RECORDS_DRIVER=local, STORAGE_DRIVER=local), including render — the native in-process HyperFrames renderer produces the MP4 on your machine at no charge. No AWS is touched. Multiple forks can be edited at once.

Flags: --port (default 3000), --dir (default ./.vidfarm-local), --key (bootstrap/browser key, or VIDFARM_API_KEY), --fork <id>, --no-open. vidfarm <template_id> is an alias for serve <template_id>.

Beyond the local editor, the devcli also wraps the entire director REST flow as 1:1 commands — discover, inspiration-add, fork, decompose, snapshot, render, approve, schedule, posts, upload/download, clips (see below), tts/stt (see below), plus a raw vidfarm api <METHOD> <path> passthrough. Each maps to exactly one REST route and prints the prod frontend URL to open. Run vidfarm --help for the full surface, or see SKILL.director.md § "vidfarm-devcli — full command surface".

Clip hunting: vidfarm clips

Mine a long-form video (podcast, VOD, webinar, YouTube URL) into tagged, searchable short clips — local-first: local ffmpeg for compute and your local claude/codex CLI subscription for scene evaluation, so it works with zero API keys. Provider keys (--provider gemini|openai|openrouter) are the fallback; --cloud is the explicit backup that runs the hunt on the deployed pipeline instead (bills AWS compute only — the AI runs on your saved BYOK keys, and the uploaded original auto-deletes from your temp folder after 30 days).

# hunt only 12:30–15:45, ~30s clips (soft 20–40s band), cropped vertical, no on-screen text
vidfarm clips scan ./podcast.mp4 --range "12:30-15:45" --duration 30 --aspect vertical --no-text \
  --prompt "guest reaction moments"

vidfarm clips search "confused reaction after reading a message"
vidfarm clips export <clip-ids…> --to ./picks

# backup: run it in the cloud (or hunt a URL without downloading anything)
vidfarm clips scan --cloud --url "https://youtube.com/watch?v=…" --duration 30 --aspect 9:16

Duration is a soft range (10→5–20s, 30→20–40s), --range means only those windows are decoded (the cost saver on long videos), and --no-text selects text-free scenes — caption removal (GhostCut) stays a per-finished-clip primitive, never a long-form pass. Run vidfarm clips --help for everything (list, match, preset, sources, export).

Speech: vidfarm tts / vidfarm stt

Text-to-speech and speech-to-text, local-first like clips: both default to your own OPENAI_API_KEY / GEMINI_API_KEY / OPENROUTER_API_KEY (no cloud job, no wallet), with --cloud as the explicit backup that runs the platform primitive on your saved BYOK keys instead. (Speech can't ride the local claude/codex agent — no audio I/O — so the local path always needs a provider key.)

# text → narration audio; the VOICE STYLE is promptable
vidfarm tts "Welcome back. Today we test the three most viral hooks." \
  --style "energetic UGC creator, conversational" --voice coral --out narration.mp3

# video or audio → transcript, BOTH formats in one pass:
#   simple subtitles (transcript.txt + timed transcript.srt)
#   + multi-speaker segments (transcript.json: {speaker, start_sec, end_sec, text})
vidfarm stt ./podcast.mp4 --out transcript          # local ffmpeg demux + your key
vidfarm stt "https://cdn.example.com/clip.mp4" --cloud   # backup: POST /api/v1/primitives/audio/transcribe

Speaker labels (diarization, on by default) need a Gemini key — OpenAI/OpenRouter keys degrade to a plain single-speaker transcript. The matching REST primitives are POST /api/v1/primitives/audio/speech and /audio/transcribe (aliases /tts, /stt).

Crash telemetry & opt-out

The devcli can report unexpected crashes (bugs — not bad args, HTTP 4xx, or network blips) to Sentry so they can be fixed. It is off unless a dedicated CLI Sentry DSN is configured (VIDFARM_DEVCLI_SENTRY_DSN), reports crash-only, and scrubs home-dir paths, API keys, and secrets before anything leaves your machine (PII off). To opt out entirely:

export VIDFARM_TELEMETRY=off   # or DO_NOT_TRACK=1