framefetch

Tiny zero-dependency client for the FrameFetch video data API.

Any social-video URL → answers, transcript, metadata, insights, and parametric frames in one call. Built for AI agents (REST + MCP), with pay-per-call or x402 (USDC) billing. Supports YouTube, YouTube Shorts, TikTok, Instagram Reels, Pinterest, and Reddit.

npm install framefetch

Requires Node 18+ (uses the built-in fetch). Get a free API key: framefetch.net.

Ask a question — get an answer, not a transcript dump

The flagship endpoint: a direct question about a video returns a short, grounded answer with timestamped quotes — instead of you having to parse a 25,000-token transcript yourself.

import { FrameFetch } from 'framefetch';

const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY });

const { ask } = await ff.ask(
  'https://www.youtube.com/watch?v=jNQXAC9IVRw',
  'What does the presenter say to do first?',
);

console.log(ask.answer);          // short, direct answer
console.log(ask.confidence);      // 'high' | 'medium' | 'low'
for (const q of ask.quotes) {     // verbatim, timestamped supporting quotes
  console.log(`[${q.t_sec}s] ${q.text}`);
}
console.log(ask.coverage);        // which part of the transcript was analyzed

$0.0075/question, charged only when an answer is actually produced. A repeat question about an already-extracted video reuses the cached transcript under the hood, so it answers fast without a re-download or re-transcription — but the answer itself is always freshly generated, never cache-served.

Frames-based answers: When a video has no transcript (e.g. Pinterest, or transcription failed), the answer is instead grounded in sampled keyframe images. In that case, coverage.mode reads "frames", quotes is always [] (no transcript text to quote), and confidence is capped at "medium" (images are weaker evidence than real transcripts).

Quick start

import { FrameFetch } from 'framefetch';

const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY });

// Everything in one call
const r = await ff.extract({
  url: 'https://www.youtube.com/watch?v=jNQXAC9IVRw',
  fields: ['metadata', 'transcript', 'frames'],
  frames: { mode: 'fps', fps: 1, width: 480 },
});
console.log(r.metadata.title, r.transcript.text, r.frames.length);

Scoped helpers

await ff.metadata(url);          // title, author, duration, views, likes…
await ff.transcript(url);        // captions, else Whisper
await ff.frames(url, { mode: 'fps', fps: 1, width: 512 });
await ff.ask(url, 'What product is being reviewed?'); // grounded Q&A, see above
await ff.digest(url);            // LLM summary of the transcript
await ff.audioDigest(url, { voice: 'nova' }); // spoken mp3 briefing (signed URL, 24h)
await ff.structured(url);        // chapters/entities/products/claims/key_moments
await ff.comments(url, { comments_cap: 50 }); // top-level comments
await ff.commentSentiment(url);  // aggregated audience-mood rollup (+ the comments)
await ff.platforms();            // capability matrix (no key)
await ff.status();               // live service health (no key)

Every helper above is a thin wrapper over extract(), so anything extract() accepts (translate, format, extra fields, …) can be passed as the last argument and is forwarded unchanged.

Search for videos, extract many at once

// Find something to extract when you don't have a URL yet
const s = await ff.search('how to make sourdough', { limit: 5 });
for (const hit of s.results) {
  console.log(hit.title, hit.url, hit.durationSec);
}

// Then extract up to 10 of them in ONE call. Shared options apply to every url.
const b = await ff.batch(s.results.slice(0, 3).map((r) => r.url), {
  fields: ['metadata', 'digest'],
});
for (const item of b.results) {
  if (!item.ok) { console.error(item.url, item.error?.code); continue; }
  console.log(item.metadata.title, '→', item.digest.gist);
}

One failing URL never fails the batch — each entry carries its own ok flag and, when ok is false, an error with code/message/hint. Per-URL frames specs are not accepted in a batch; use extract() for those.

Translate the transcript, export subtitles

// translate the transcript into 1 of 25 languages (surfaced as transcript_translated)
const r = await ff.transcript(url, { translate: 'ja' });
console.log(r.transcript_translated.text);

// export subtitles directly — format is sent as a query param, response comes back as a string
const srt = await ff.transcript(url, { format: 'srt' });         // source-language subtitles
const vttJa = await ff.transcript(url, { translate: 'ja', format: 'vtt' }); // translated subtitles

Digest, audio digest, structured JSON, comments

const r = await ff.extract({
  url,
  fields: ['digest', 'audio_digest', 'structured', 'comments', 'comment_sentiment'],
  voice: 'nova',        // spoken voice for audio_digest (alloy/echo/fable/onyx/nova/shimmer/Fritz-PlayAI)
  comments_cap: 50,     // cap on top-level comments fetched (1-200, default 100; YouTube only)
});

r.digest.gist;                    // LLM summary of the transcript
r.audio_digest.url;               // signed mp3 URL of the spoken briefing (24h TTL)
r.structured.chapters;            // chapters/entities/products/claims/key_moments (vision LLM)
r.comments.items;                 // top-level comments
r.comment_sentiment.summary;      // aggregated audience-mood rollup

No signup needed

const ff = new FrameFetch();          // no key
await ff.demo('https://youtu.be/jNQXAC9IVRw');   // instant metadata, rate-limited
const { key } = await ff.createKey('[email protected]'); // self-serve key + free credit

Errors

Failed calls throw FrameFetchError with .status, .code, and .hint:

import { FrameFetchError } from 'framefetch';
try {
  await ff.transcript(url);
} catch (e) {
  if (e instanceof FrameFetchError && e.status === 402) {
    // out of credit — top up at https://framefetch.net or via x402
  }
}

Options

new FrameFetch({
  apiKey: '…',                 // or FRAMEFETCH_API_KEY env var
  baseUrl: 'https://framefetch.net',
  timeoutMs: 120_000,
  fetch: customFetch,          // inject your own fetch if needed
});

Full extract() request shape

ff.extract({
  url: string,
  fields?: Field[],            // 'metadata' | 'insights' | 'transcript' | 'frames' | 'text_overlay'
                                // | 'digest' | 'audio_digest' | 'structured' | 'comments'
                                // | 'comment_sentiment' | 'delta'
  frames?: { mode, n, fps, from, to, format, width },
  translate?: string,          // ISO-639-1 target language (25 supported)
  voice?: string,               // TTS voice for audio_digest
  comments_cap?: number,       // 1-200, default 100
  ask?: string,                 // 3-500 char question — see ff.ask() above
  publish?: boolean,            // opt in to a public per-video SEO page
  format?: 'md' | 'markdown' | 'srt' | 'vtt', // alternate egress rendering (returns a string, not JSON)
});

See index.d.ts for the complete typed response shape (ExtractResult, Ask, VideoStructured, VideoComments, CommentSentiment, AudioDigest, …).

MCP

Prefer MCP? FrameFetch also ships an MCP server at https://framefetch.net/mcp — see the docs.

MIT licensed.