@jtdigital/renderbox-sdk

Typed TypeScript SDK for renderbox video processing. You describe a pipeline as a graph of sort-typed streams; renderbox's GPU/CPU workers run it. The SDK builds the graph and the job message — it does not process video itself.

npm install @jtdigital/renderbox-sdk

Building a pipeline

Open inputs, chain operations on the returned streams, attach outputs:

import { graph } from '@jtdigital/renderbox-sdk';

const g = graph();
const [v, a] = g.open('in.mp4');               // [VideoStream, AudioStream]

const out = v.scale({ w: 1280, h: 720 })       // ops are methods → autocomplete
             .fade({ type: 'in', d: 1 });

g.write('out.mp4', out, a);

Streams are sort-typed: VideoStream exposes video ops, AudioStream audio ops, DetectionStream tracking and filtering. A sort mismatch (piping audio into a video op) is a compile error, not a runtime one.

Detection → tracking → redaction

const g = graph();
const [v, a] = g.open('street.mp4');

const faces = v
  .detect({ model: 'scrfd_10g', threshold: 0.12 })   // VideoStream → DetectionStream
  .track({ algorithm: 'bytetrack' });                // stable ids across frames

g.write('redacted.mp4', v.redact(faces, { mode: 'black' }), a);

Multiple inputs

Combine ops are methods on the first stream; open more inputs on the same graph:

const g = graph();
const [v, a] = g.open('clip.mp4');
const [logo]  = g.open('logo.png');

g.write('branded.mp4', v.overlay(logo, { x: 20, y: 20 }), a);

Detections to a data sink

Not every output is a video — write structured results too:

const g = graph();
const [v] = g.open('traffic.mp4');
const cars = v.detect({ model: 'yolo11x', classes: 'car' }).track({ algorithm: 'bytetrack' });
g.writeDetections('cars.jsonl', cars);

Recipes

One-liners for common pipelines. faceRedact, depthBlurPipeline, and makeSlideshow return a ready Graph; letterbox and snip are stream transforms:

import { faceRedact, snip } from '@jtdigital/renderbox-sdk';

const g = faceRedact('in.mp4', 'out.mp4', { redactMode: 'blur' });

// or as a step in your own graph:
const [v] = graph().open('long.mp4');
const clip = snip(v, 30, 45);   // seconds 30–45

Running your pipeline on renderbox

A Graph is a description. To actually render it, submit it as a job to renderbox infrastructure — the orchestrator compiles it and dispatches it to a GPU or CPU worker, which streams back progress and a completion result.

You reference inputs and outputs by S3 key. Two calls give you everything the worker needs: g.json() is the graph_ir, and g.io() is the S3 ⇄ storage binding (which object to download for each input, where to upload each output). The SDK assigns no identity — the job id, tenant, and user come from whoever submits (your app/auth, or the managed client).

import { graph, type ProgressMessage, type CompletionMessage } from '@jtdigital/renderbox-sdk';

const g = graph();
const [v, a] = g.open('uploads/raw.mp4');           // reference inputs by S3 key
g.write('renders/out.mp4', v.scale({ w: 1920, h: 1080 }), a);

const io = g.io();
// io.inputs  → [{ s3_key: 'uploads/raw.mp4', filename: 'raw.mp4', content_type: 'video/mp4' }]
// io.outputs → [{ s3_key: 'renders/out.mp4', filename: 'out.mp4' }]

// Assemble the job with identity from YOUR context, then publish over AMQP.
// (The message omits pipeline_id so the orchestrator routes on graph_ir.)
const job = {
  job_id: myApp.newJobId(),      // assigned by your app/DB — not the SDK
  tenant_id: ctx.tenantId,       // from auth
  graph_ir: g.json(),
  input_files: io.inputs,
};
await channel.publish(EXCHANGE, ROUTING_KEY, Buffer.from(JSON.stringify(job)));

You never write filenames by hand: open('uploads/raw.mp4') gives the graph a sandbox-safe basename (raw.mp4) and records the S3 key for you — the worker rejects slashes in the sandbox, which is why the graph and the binding differ.

Then consume results off your result queue. Progress arrives as the render proceeds; completion carries the output location(s):

function onProgress(msg: ProgressMessage) {
  // percent is null until it can be computed (null ≠ 0%)
  console.log(`${msg.job_id}: ${msg.percent ?? '—'}% @ ${msg.fps} fps`);
}

function onCompleted(msg: CompletionMessage) {
  if (msg.status === 'success') {
    for (const out of msg.outputs) console.log(out.kind, out.s3_key, out.bytes);
  }
}

Prefer a managed client?

@jtdigital/renderbox-client wraps the renderbox SaaS HTTP API — API-key auth, asset upload, job submission, SSE progress streaming, and result download — so you can hand it a Graph and get a finished render back without running your own transport.

Surface

• graph() → Graph — inputs: open / openVideo / openAudio / rtsp / hls / mqtt / camera / screenCapture / slot, plus synthetic sources. Outputs: write / writeVideo / writeAudio / writeDetections / writeTranscript / writeMqtt / writeHttp / outputSlot. Terminals: json() (graph_ir) and io() (S3 ⇄ sandbox bindings).
• Stream methods — 220+ typed operations, one per engine op, on the sort it applies to (scale, crop, fade, detect, track, transcribe, overlay, redact, depthBlur, …).
• Recipes — letterbox, snip, faceRedact, depthBlurPipeline, makeSlideshow.
• Wire types — GraphJSON, OpNode, JobMessage, ProgressMessage, CompletionMessage, FailureMessage.

License

Apache-2.0 © JT Digital d.o.o.