@trillboards/edge-sdk

Turn any screen into an AI-powered DOOH advertising endpoint. Edge AI audience sensing, ad delivery, federated learning, and cloud inference in one package. Linux and Windows.

Quick Start

npm install @trillboards/edge-sdk
npx trillboards-edge download-models
npx trillboards-edge start --config trillboards.config.json

What This Does

The Edge SDK runs on digital signage hardware (kiosks, vending machines, billboards, CTV devices) and provides:

• Audience sensing — on-device face detection, gaze tracking, emotion analysis, foot traffic estimation via ONNX models (no cloud dependency)
• Ad delivery — Chromium browser lifecycle, display management, VAST/VPAID ad rendering
• Federated learning — privacy-preserving on-device model training with sparse gradient uploads
• Cloud inference — Gemini Vision scene analysis, content moderation, contextual targeting
• Device management — heartbeat, health monitoring, auto-recovery, signal buffering for offline resilience

All audience data stays on-device. Only aggregated, anonymized metrics are transmitted.

Configuration

Create trillboards.config.json:

{
  "apiBaseUrl": "https://api.trillboards.com",
  "deviceToken": "your-device-token",
  "camera": { "enabled": true, "width": 640, "height": 480, "fps": 10 },
  "audio": { "enabled": true },
  "models": { "dir": "./models", "executionProvider": "cpu" },
  "kiosk": { "enabled": true, "url": "https://screen.trillboards.com" },
  "cloud": { "enabled": false },
  "federated": { "enabled": true },
  "logLevel": "info"
}

Sensing cadence (cloudSensing.motionGate)

Every key is optional and every default is derived rather than typed in — see src/connectorCadence.ts. The two cadences are separate and are bounded by different hardware, which is the thing to understand before overriding either:

| Key | What it controls | Default | |---|---|---| | scanIntervalMs | How often a feed is LOOKED at. Bounded by the recorder — one RTSP grab costs ~2 s (DESCRIBE/SETUP/PLAY plus a wait for a keyframe), and a recorder takes 2 concurrent grabs. | lookIntervalMsForFeedCount(feeds) — 2 s for a small install, 16 s for 16 feeds on one recorder | | minScanIntervalMs | Floor under scanIntervalMs. A units-typo guard: scanIntervalMs: 5 means 200 looks/second/feed. 0 disables it. The clamp is reported in getSensingHealth().cascade.scanClamped, never applied silently. | 200 (Frigate's 5 fps) | | minAnalyzeIntervalMs | Floor between two motion-triggered ANALYSES of one feed. Bounded by the box — one local inference is seconds, on a single admission slot. | faceTriggeredIntervalMs (30 s) | | heartbeatIntervalMs | The still-scene reporting floor, so "nothing is happening" stays distinguishable from "the connector died". | periodicIntervalMs | | stationaryReanalyzeAfterMs | How long same-place motion — a parked car, a swinging sign, a display — is suppressed before one verdict is let through. A duration, not a frame count: a count re-interprets itself every time the look cadence moves, and at a 2 s scan a frame count of 10 costs 9x the analyses it costs at 30 s. 0 disables suppression. | 300000 | | stationaryReanalyzeEvery | Deprecated. The retired frame count. Still honoured — it is multiplied by the resolved scan interval, at the one place that knows both — so an existing config keeps meaning what it meant. Prefer the duration. | — |

Catch rate per event shape is a gated number, not a claim: see docs/runbooks/sensing-catch-rate.md.

Programmatic Usage

import { EdgeAgent, loadConfig } from '@trillboards/edge-sdk';

const config = loadConfig('trillboards.config.json');
const agent = new EdgeAgent(config);

agent.on('started', ({ fingerprint, tier }) => {
  console.log(`Device ${fingerprint} running at TIER_${tier}`);
});

await agent.start();

CLI Commands

| Command | Description | |---------|-------------| | trillboards-edge demo | Try it instantly -- synthetic audience sensing with a live terminal dashboard. No token, camera, or models required. | | trillboards-edge init | Interactive setup wizard -- prompts for token, platform, camera, audio, kiosk, models, and execution provider | | trillboards-edge start [--config path] | Start the edge agent | | trillboards-edge status [--host h] [--port p] [--json] [--watch] | Query the agent status endpoint and display a formatted dashboard | | trillboards-edge diagnose [--json] | Run 10 system diagnostic checks (Node.js, platform, memory, CPU, disk, camera, models, API, GPU, network) | | trillboards-edge mcp | Start the MCP server for AI agent integration (Claude Desktop, Cursor) | | trillboards-edge download-models [--dir path] | Download ONNX models (BlazeFace, YAMNet) | | trillboards-edge version | Print SDK version |

See docs/CLI.md for the full CLI reference with all flags and configuration details.

Capability Tiers

The SDK auto-detects device capabilities and adjusts inference accordingly:

| Tier | RAM | NPU | Models | FPS | |------|-----|-----|--------|-----| | TIER_1 | < 2 GB | No | Face only | 2 | | TIER_2 | 2-4 GB | No | Face + Audio | 5 | | TIER_3 | 4-8 GB | Optional | All models | 10 | | TIER_4 | 8+ GB | Yes | All + high-res | 15+ |

Execution Providers

• cpu — works everywhere (default)
• openvino — Intel integrated GPU (2-3x faster)
• directml — Windows GPU acceleration
• cuda — NVIDIA GPU

Acoustic Sensing (opt-in)

The connector can hear as well as see. A microphone catches what a camera has nothing to show for — a freezer door left open behind a shelf, glass breaking off frame, an alarm in the next room — so an acoustic onset is allowed to carry a frame that the motion cascade would otherwise have dropped as no_motion.

It is OFF by default, and that default is a consent position rather than a rollout setting. Many US states are all-party-consent for audio; the cameras this connector attaches to belong to the customer and were very likely installed, signed and papered for video. Turning this on has to be somebody's explicit decision.

new CloudSensingManager({
  loops,
  motionGate: { enabled: true },      // the acoustic gate rides the visual cascade
  audioSensing: {
    enabled: true,                    // <- explicit; default false
    maxAudioSessionsPerSource: 1,     // long-lived audio sessions per recorder
  },
});

Every failure on this path is non-fatal by construction — a recorder that refuses admission, a stream with no audio track, a missing ffmpeg — all leave the camera loops untouched. Audio is supplementary and can never stop a feed from producing frames.

Why the gate is DSP and not the on-device model

Gemma 4 was measured against this exact runtime before the path was built. It transcribes speech excellently and does not name acoustic events: on real recordings it scored 2/8 on an open prompt and 3/8 forced-choice, where every hit was an animate vocalization (dog, baby, applause) and every mechanical / impact / tonal event — breaking glass, a door slam, a smoke alarm — missed, defaulting to the label speech on audio containing no speech.

So the two halves are split by what each is actually good at:

| Job | Implementation | |---|---| | "Did something happen?" | AudioOnsetDetector — rolling background estimate + threshold | | "What was said?" | LiteRtVlmProvider.transcribeAudio (Gemma, via litert-lm input_audio) |

The full measured battery is in the AudioOnsetDetector header, and the operator view is in docs/runbooks/audio-sensing-substrate.md §0.

New exports

| Export | Purpose | |---|---| | AudioOnsetDetector, rmsDbfs, SILENCE_DBFS | Acoustic onset detection (dBFS — not a 0..4 tier) | | attachAudioOnset | Joins an audio transport to a feed's MotionGate; returns null when a feed has no audio | | encodeWav16, takeMostRecentSeconds | Float32 PCM → 16-bit mono WAV for the model's input_audio | | MotionGateReason: 'audio_onset' | New cascade verdict — a heard event carried this frame |

Sub-Packages

This is the umbrella package. For fine-grained control, install sub-packages individually:

| Package | Purpose | |---------|---------| | @trillboards/edge-core | Device identity, Socket.io, config, signal buffer, health monitoring | | @trillboards/edge-sensing | ONNX face detection, audio classification, audience metrics | | @trillboards/edge-ads | Chromium browser lifecycle, display management, ad delivery | | @trillboards/edge-federated | On-device federated learning, sparse gradient upload | | @trillboards/edge-cloud | Gemini Vision scene analysis, contextual targeting | | @trillboards/edge-platform-linux | V4L2 camera, PulseAudio, systemd, kiosk mode | | @trillboards/edge-platform-windows | DirectShow, RTSP IP cameras, WASAPI audio | | @trillboards/edge-platform-rtsp | RTSP camera frames, RTSP audio sessions, macOS microphone |

MCP Server (AI Agent Integration)

The SDK includes a built-in MCP (Model Context Protocol) server that allows AI agents to discover, configure, and diagnose edge devices.

trillboards-edge mcp

The server communicates over stdio using JSON-RPC 2.0 and talks to the running edge agent's StatusServer on localhost:9090.

7 tools available: get-device-status, get-audience-live, configure-sensing, run-benchmark, diagnose-hardware, list-models, get-buffer-stats.

Claude Desktop config (claude_desktop_config.json):

{
  "mcpServers": {
    "trillboards-edge": {
      "command": "npx",
      "args": ["-y", "@trillboards/edge-sdk", "mcp"],
      "env": { "EDGE_STATUS_PORT": "9090" }
    }
  }
}

Cursor config (.cursor/mcp.json):

{
  "mcpServers": {
    "trillboards-edge": {
      "command": "npx",
      "args": ["-y", "@trillboards/edge-sdk", "mcp"]
    }
  }
}

See docs/MCP.md for the full MCP server reference.

Status Endpoint

When the edge agent is running, it exposes an HTTP status server (default port 9090) with three endpoints:

| Endpoint | Method | Description | |----------|--------|-------------| | /status | GET | Full JSON status payload (device, sensing, subsystems, buffer) | | /health | GET | Health check -- returns 200 if healthy, 503 if degraded | | /metrics | GET | Prometheus-format metrics for scraping |

curl http://localhost:9090/status    # Full status JSON
curl http://localhost:9090/health    # Health check
curl http://localhost:9090/metrics   # Prometheus metrics

Query from the CLI:

trillboards-edge status                     # Formatted dashboard
trillboards-edge status --watch             # Refresh every 5s
trillboards-edge status --host 192.168.1.50 # Remote device
trillboards-edge status --json              # Raw JSON

See docs/STATUS_API.md for the full status API reference.

Docker Deployment

# CPU (default)
docker run -d \
  --name trillboards-edge \
  -e DEVICE_TOKEN=your-token \
  -p 9090:9090 \
  --device /dev/video0:/dev/video0 \
  ghcr.io/trillboards/edge-sdk:latest

# NVIDIA GPU
docker run -d \
  --name trillboards-edge \
  --gpus all \
  -e DEVICE_TOKEN=your-token \
  -p 9090:9090 \
  --device /dev/video0:/dev/video0 \
  ghcr.io/trillboards/edge-sdk:cuda

docker-compose:

services:
  edge-agent:
    image: ghcr.io/trillboards/edge-sdk:latest
    restart: unless-stopped
    environment:
      - DEVICE_TOKEN=${DEVICE_TOKEN}
    ports:
      - "9090:9090"
    devices:
      - /dev/video0:/dev/video0
    volumes:
      - ./models:/app/models
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9090/health"]
      interval: 30s
      timeout: 5s
      retries: 3

The create-trillboards-edge scaffolding tool can generate a complete Docker or Kubernetes project for you:

npx create-trillboards-edge my-deploy --template docker
npx create-trillboards-edge fleet --template kubernetes

See deploy/docker/README.md and deploy/helm/README.md for full deployment guides.

Platform Support

• Linux -- Ubuntu 20.04+, Debian 11+, Raspberry Pi OS (ARM64)
• Windows -- Windows 10/11 (x64)

Get a Device Token

npx @trillboards/ads-sdk init

Or register via the Trillboards API.

Documentation

| Document | Description | |----------|-------------| | docs/CLI.md | Full CLI reference with all commands, flags, and configuration | | docs/MCP.md | MCP server reference for AI agent integration | | docs/STATUS_API.md | Status endpoint API reference | | deploy/macos/README.md | macOS one-click .pkg installer (local-inference CCTV connector) | | deploy/docker/README.md | Docker deployment guide | | deploy/helm/README.md | Helm chart deployment guide | | create-trillboards-edge | Project scaffolding tool |

License

MIT