flockml
v1.3.0
Published
Developer Grade Decentralized AI Node Engine (Wasm Powered)
Readme
⚡ The Paradigm Shift
The Problem: Training deep neural networks requires renting astronomical A100 GPU clusters from AWS or GCP. Furthermore, user data must be shipped across the internet to centralized servers, creating massive privacy bottlenecks.
The Solution: flockml drops a silent, non-blocking Web Worker into your Next.js/React app. When a user visits your site, their local browser processes tiny batches of training data using our blazingly fast Rust WebAssembly Engine.
- $0 Compute Costs: 10,000 visitors = 10,000 active Edge GPUs.
- 97% Native C++ Speeds: Powered by our custom
flockml-wasmRust bridge. - Cryptographic Privacy: Data never leaves the browser. We inject Laplacian noise into gradients before network transmission.
- Infinite Scalability: Your compute power scales linearly with your web traffic.
📦 Installation
Install the open-source core via NPM:
npm install flockml🚀 Quickstart Guide
FlockML uses a strict Coordinator-Node architecture. You need a central Node.js server (The Coordinator) and clients (The Browsers).
1. The Client (React / Next.js / Vanilla JS)
The client node initializes the Rust Wasm engine, processes the math without blocking the UI thread, and sends the quantized gradients back to your server.
import { FlockNode } from 'flockml';
async function bootEdgeNode() {
// 1. Initialize the Node (2 inputs, 4 hidden, 1 output)
const node = new FlockNode(2, 4, 1);
// 2. Boot the High-Performance Rust WebAssembly Engine
await node.initEngine();
// 3. Connect to your central Coordinator
node.connect('wss://api.yourdomain.com/flock');
// 4. Train a local batch asynchronously (non-blocking 60fps)
const inputs = [[0, 0], [0, 1], [1, 0], [1, 1]];
const targets = [[0], [1], [1], [0]];
await node.trainLocalBatchAsync(inputs, targets);
// 5. Encrypt, Quantize (Int8), and Export
const securePayload = node.exportSecureGradients();
console.log("Ready for transmission:", securePayload);
}2. The Server (Node.js)
The Coordinator simply receives the 8-bit payloads from millions of browsers, decompresses them, and applies standard Federated Averaging.
import { Coordinator } from 'flockml';
const server = new Coordinator();
// Example: Receiving payloads via WebSocket
server.on('payload', (qWeightsIH, qWeightsHO, qBiasH, qBiasO) => {
server.aggregate(qWeightsIH, qWeightsHO, qBiasH, qBiasO);
const globalModel = server.getGlobalWeights();
console.log("Global Model Updated!");
});⚙️ Architecture Under the Hood
To make deep learning run natively in the browser at enterprise speeds, FlockML implements three core optimizations:
The Rust Wasm Bridge (
flockml-wasm) JavaScript is too slow for billions of matrix dot-products. FlockML compiles raw Rust down to WebAssembly, allocating flat memory arrays at the browser's absolute hardware limit.8-Bit Quantization (
Quantizer) Standard ML frameworks (PyTorch) rely on heavy 32-bit floating point (Float32) numbers. FlockML mathematically compresses weight matrices down toInt8payloads. This reduces WebSocket payload sizes by 4x and dramatically speeds up local dot-products on mobile processors.Differential Privacy (
DifferentialPrivacy) FlockML utilizes Cryptographic Laplacian Noise to anonymize the gradients extracted from the user's local data. Because of the mathematics of Federated Averaging (FedAvg), when the central server aggregates thousands of these noisy gradients, the noise cancels out to 0, leaving only the pure, learned signal.
🛠 Project Structure
src/client-node.ts- Client API wrapper and Wasm orchestratorsrc/coordinator.ts- FedAvg central server aggregationsrc/quantization.ts- Float32 -> Int8 payload compressionsrc/privacy.ts- Differential Privacy via Laplacian Noiseflockml-wasm/- The Rust Cargo workspace generating the WebAssembly bindings
💼 Enterprise Support
The open-source flockml NPM package utilizes standard Federated Averaging. For proprietary Enterprise grid deployments requiring Asynchronous Decentralized SGD (AD-SGD) Staleness Penalties and Swarm Pipeline Parallelism, please contact the maintainers.
📜 License
MIT License.
