@trillboards/edge-cloud

Cloud vision inference for DOOH devices. Gemini Vision scene analysis, content moderation, contextual targeting, and demographic estimation from video frames.

Install

npm install @trillboards/edge-cloud

What This Does

Sends captured video frames to Google Gemini Vision API for advanced analysis that goes beyond on-device ONNX models:

• Scene Analysis — describes what's happening in the scene (people, objects, activities, environment)
• Contextual Targeting — recommends ad categories and bidding adjustments based on real-time scene context
• Content Moderation — evaluates frames for display safety with confidence scores and block reasons
• Crowd Density — classifies venue occupancy (empty, sparse, moderate, dense, packed)
• Frame Capture Manager — captures frames at configurable cadence, batches and encodes to JPEG/PNG

Usage

import { GeminiSceneAnalyzer, FrameCaptureManager } from '@trillboards/edge-cloud';

// Capture frames from camera feed
const capture = new FrameCaptureManager({
  cadenceSeconds: 30,
  faceDetectedCadenceSeconds: 10, // faster when faces present
  maxFramesPerBatch: 3,
  quality: 80
});

capture.on('batchReady', async (frames) => {
  const result = await analyzer.analyzeScene(frames);
  console.log(`Scene: ${result.sceneDescription}`);
  console.log(`People: ${result.peopleCount}`);
  console.log(`Objects: ${result.detectedObjects.join(', ')}`);
});

// Scene analysis via Gemini
const analyzer = new GeminiSceneAnalyzer({
  apiKey: process.env.GEMINI_API_KEY,
  model: 'gemini-2.0-flash'
});

// Contextual targeting
const targeting = await analyzer.getContextualTargeting(frames);
// → { recommendedCategories: ['retail', 'food'], relevanceScore: 0.85 }

// Content moderation
const moderation = await analyzer.moderate(frames);
// → { isSafeForDisplay: true, safetyCategory: 'safe', confidenceScore: 0.98 }

Cost Management

const cost = analyzer.getCostSnapshot();
// → { estimatedTokensPerMinute: 150, costPerDay: 0.45, costPerMonth: 13.50 }

Scene Analysis Result

interface SceneAnalysisResult {
  sceneDescription: string;
  peopleCount: number;
  detectedObjects: string[];
  activities: string[];
  confidence: number;
}

When to Use This vs. Edge Sensing

| Capability | @trillboards/edge-sensing | @trillboards/edge-cloud | |-----------|--------------------------|------------------------| | Face detection | On-device ONNX | N/A | | Scene understanding | N/A | Gemini Vision | | Cost | Free (local compute) | Per-API-call | | Latency | < 50ms | 500ms-2s | | Privacy | No data leaves device | Frames sent to Google |

Use edge-sensing for real-time audience metrics. Use edge-cloud for rich scene context and content moderation.

License

MIT