Stream Lego 🧱 | Distributed Edge AI WebRTC Framework

Stream Lego is a highly scalable, WebRTC-based framework designed for building complex Distributed Edge AI topologies and DAG (Directed Acyclic Graph) streaming architectures. Built as an elegant wrapper over webrtc-tree, it allows developers to split, process, aggregate, and merge media streams across multiple edge nodes seamlessly.

If you are building real-time AI agents, live translation services, edge-computed video filters, or decentralized podcasting platforms, Stream Lego is your ultimate networking backbone.

Features

• Distributed Edge Computing: Offload heavy AI inference (Voice Recognition, Computer Vision) from single servers to distributed, low-cost edge nodes.
• WebRTC DAG Topology: Move beyond standard 1-to-Many broadcasting. Create complex WebRTC pipelines using Virtual Rooms.
• Scatter and Merge: Extract audio and video tracks independently, process them on dedicated GPUs/CPUs, and perfectly merge them back together in real-time.
• Zero-Config Routing: Inherits the ultra-stable, self-healing BFS parent-assignment mesh networking from webrtc-tree.

Internal Architecture

How stream-lego orchestrates multiple WebRTC trees without altering the core:

classDiagram
    class StreamLegoCoordinator {
        -Record<string, RTCTreeCoordinator> virtualRooms
        +createVirtualRoom(virtualRoomId, config)
        +handleSignaling(virtualRoomId, from, to, payload)
    }

    class RTCTreeCoordinator {
        <<webrtc-tree Server>>
        +getAssignedParent()
    }

    class StreamLegoClient {
        -Record<string, RTCTreeClient> inputs
        -Record<string, RTCTreeClient> outputs
        +addInput(config)
        +addOutput(config, stream)
    }

    class RTCTreeClient {
        <<webrtc-tree Client>>
        +connectToMesh()
    }

    StreamLegoCoordinator "1" *-- "many" RTCTreeCoordinator : Manages Rooms
    StreamLegoClient "1" *-- "many" RTCTreeClient : Manages Sub-Clients

The Concept of Virtual Tags

In modern AI live-streaming, a single edge node cannot handle all AI workloads simultaneously. Stream Lego solves this by introducing Tags to distribute the workload:

• N (Normal / Viewer): The original broadcaster or the end viewer.
• A (Audio Processor): An edge node that solely receives audio, processes it (e.g., AI noise cancellation, speech-to-text), and broadcasts the result.
• B (Video Processor): An edge node that solely receives video, processes it (e.g., AI avatars, object detection), and broadcasts the result.
• C (Audio Aggregator): Merges multiple audio streams into one (e.g., Band Practice or remote Podcasting).
• D (Video Aggregator): Merges multiple video streams into one.
• final (Composite Merger): Subscribes to processed audio and video, perfectly synchronizes them, and becomes a Composite Initial Node to broadcast the final result to the audience.

Architectural Use Cases

1. The Distributed Edge AI Split-Merge

Offload audio processing to a CPU instance and video processing to a GPU instance, merging them for the viewers.

graph TD
    classDef streamer fill:#ff4757,stroke:#333,stroke-width:2px,color:#fff;
    classDef edgeA fill:#1e90ff,stroke:#333,stroke-width:2px,color:#fff;
    classDef edgeB fill:#2ed573,stroke:#333,stroke-width:2px,color:#fff;
    classDef finalNode fill:#9c88ff,stroke:#333,stroke-width:2px,color:#fff;
    classDef viewer fill:#a4b0be,stroke:#333,stroke-width:1px,color:#fff;

    S((Broadcaster)):::streamer

    A1((Node A<br/>AI Audio Processing)):::edgeA
    B1((Node B<br/>AI Video Filters)):::edgeB

    F((Node Final<br/>Composite Merger)):::finalNode

    V1((Viewer N)):::viewer
    V2((Viewer N)):::viewer

    S -- "Audio Track (Room: raw_audio)" --> A1
    S -- "Video Track (Room: raw_video)" --> B1

    A1 -- "Processed Audio (Room: proc_audio)" --> F
    B1 -- "Processed Video (Room: proc_video)" --> F

    F -- "Merged AV Stream (Room: final_mixed)" --> V1
    F --> V2

2. Multi-Node Audio Aggregation (Remote Band/Podcast)

Mix multiple isolated audio sources at the edge using the Web Audio API to create a unified broadcast.

graph TD
    classDef streamer fill:#ff4757,stroke:#333,stroke-width:2px,color:#fff;
    classDef edgeC fill:#ffa502,stroke:#333,stroke-width:2px,color:#fff;
    classDef viewer fill:#a4b0be,stroke:#333,stroke-width:1px,color:#fff;

    G((Guitarist)):::streamer
    S((Singer)):::streamer
    D((Drummer)):::streamer

    C((Node C<br/>Audio Aggregator)):::edgeC

    V1((Viewer)):::viewer

    G -- "Room: guitar_audio" --> C
    S -- "Room: singer_audio" --> C
    D -- "Room: drum_audio" --> C

    C -- "WebAudio Mixed Track<br/>(Room: band_mixed_audio)" --> V1

3. Real-Time AI Multi-Language Translation (One-to-Many)

A single broadcaster sends one video feed. Multiple Edge Nodes translate the audio into different languages, and Composite Final Nodes merge them for country-specific broadcasting!

graph TD
    classDef streamer fill:#ff4757,stroke:#333,stroke-width:2px,color:#fff;
    classDef edgeA fill:#1e90ff,stroke:#333,stroke-width:2px,color:#fff;
    classDef finalNode fill:#9c88ff,stroke:#333,stroke-width:2px,color:#fff;
    classDef viewer fill:#a4b0be,stroke:#333,stroke-width:1px,color:#fff;

    Streamer((Original Broadcaster)):::streamer
    
    A_JP((Node A<br/>Japanese AI Translator)):::edgeA
    A_FR((Node A<br/>French AI Translator)):::edgeA

    F_JP((Final Node JP)):::finalNode
    F_FR((Final Node FR)):::finalNode

    V_JP((JP Audience)):::viewer
    V_FR((FR Audience)):::viewer

    Streamer -- "Room: eng_audio" --> A_JP
    Streamer -- "Room: eng_audio" --> A_FR
    
    Streamer -. "Room: raw_video" .-> F_JP
    Streamer -. "Room: raw_video" .-> F_FR

    A_JP -- "Room: jp_audio" --> F_JP
    A_FR -- "Room: fr_audio" --> F_FR

    F_JP -- "Japanese Broadcast (Room: stream_jp)" --> V_JP
    F_FR -- "French Broadcast (Room: stream_fr)" --> V_FR

Installation

npm install stream-lego webrtc-tree

Quick Start

Build topologies like Lego blocks using straightforward Virtual Rooms logic:

import { StreamLegoClient } from 'stream-lego/client';

const mergerNode = new StreamLegoClient('merger-123', config);

// Subscribe to processed audio and raw video seamlessly
await mergerNode.addInput({ virtualRoomId: 'proc_audio' });
await mergerNode.addInput({ virtualRoomId: 'raw_video' });

// Merge the streams using standard Web APIs
const mergedStream = new MediaStream([ ...tracks ]);

// Output the final composite stream to the viewers
await mergerNode.addOutput({ virtualRoomId: 'final_mixed' }, mergedStream);

License

MIT