@xingm/vmap-video

v0.0.4

Published

9 days ago

Drone video timing, OSD normalization, projection geometry, and Cesium overlay SDK.

0High
0Medium
0Low

webrtc cesium video osd telemetry drone sdk

@xingm/vmap-video

无人机视频时间、OSD 标准化、时间匹配、地图投影与 Cesium 投射 SDK。

文档导航

API 文档：当前文件
对接手册：docs/xingm-vmap-video-integration.md

安装

pnpm add @xingm/vmap-video

如果使用 Cesium 投影能力：

pnpm add cesium

如果使用播放器辅助组件：

pnpm add vue

导出入口

@xingm/vmap-video：聚合导出核心类型、session、telemetry、sync、projection、cesium
@xingm/vmap-video/session
@xingm/vmap-video/telemetry
@xingm/vmap-video/adapters
@xingm/vmap-video/sync
@xingm/vmap-video/osd
@xingm/vmap-video/webrtc-time
@xingm/vmap-video/projection
@xingm/vmap-video/cesium
@xingm/vmap-video/player

核心类型

interface VmapCornerPoint {
  lat: number
  lon: number
}

interface VmapTelemetryFrame {
  frame: number
  timestampMs: number
  lat: number
  lon: number
  alt: number
  agl: number
  rollDeg: number
  pitchDeg: number
  yawDeg: number
  gimbalYawDeg?: number
  gimbalPitchDeg: number
  hfovDeg: number
  vfovDeg: number
  frameCenterLat: number
  frameCenterLon: number
  corners?: VmapCornerPoint[]
  raw?: unknown
}

字段说明：

timestampMs：Unix 毫秒时间戳
corners：视频画面四角地理坐标；存在时投影优先直接使用
raw：保留原始消息，便于排障

推荐主入口：`session`

如果你的目标是“实时视频 + 实时 OSD + Cesium 投影”，优先使用 createFusionSession。

`createFusionSession`

import { createFusionSession } from '@xingm/vmap-video'

interface FusionSessionOptions<TFrame = VmapTelemetryFrame> {
  telemetry?: FusionSessionTelemetryOptions<TFrame>
  videoClock?: {
    clockRate?: number
  }
  renderer?: VmapProjectionRendererAdapter | null
  realtimeProjection?: RealtimeProjectionOptions
}

`telemetry`

interface FusionSessionTelemetryOptions<TFrame = VmapTelemetryFrame> {
  transport:
    | ({ type: 'websocket' } & WebSocketTelemetryTransportOptions)
    | TelemetryTransport
  protocol?: TelemetryProtocolAdapter<TFrame>
  matchOptions?: TimeMatchOptions
  autoConnect?: boolean
}

transport：遥测输入通道
protocol：原始消息解析适配器；不传时默认使用 createXingmTelemetryAdapter()
matchOptions：时间匹配阈值与缓冲大小
autoConnect：是否在 start() 时自动连接遥测

`renderer`

一般传 createCesiumProjectionRenderer(...) 的返回值。

`realtimeProjection`

interface RealtimeProjectionOptions {
  enabled?: boolean
  adaptiveOffset?: {
    enabled?: boolean
    allowedSources?: string[]
    maxOffsetMs?: number
    resetThresholdMs?: number
    smoothingFactor?: number
  }
  match?: {
    thresholdMs?: number
    releaseGraceMs?: number
  }
  stableFrameHold?: {
    enabled?: boolean
    projectablePitchRange?: [number, number]
  }
  geometryUpdate?: {
    minIntervalMs?: number
    positionEpsilonMeters?: number
  }
  cornerOrdering?: {
    enabled?: boolean
    yawStrategy?: 'gimbal-first' | 'body-plus-gimbal' | 'body-only'
  }
}

用途：

adaptiveOffset：实时视频时间补偿
match：时间匹配阈值与 grace 窗口
stableFrameHold：投影稳定帧保持
geometryUpdate：投影几何刷新频率控制
cornerOrdering：输入四角点顺序规范

Session 实例方法

interface FusionSession<TFrame = VmapTelemetryFrame> {
  start(): Promise<void>
  stop(): void
  destroy(): void
  connectTelemetry(): Promise<void>
  disconnectTelemetry(): void
  attachVideo(video: VmapVideoSourceElement): void
  updateVideoTime(videoTimestampMs: number | null, source?: string): void
  updateVideoTimeFromRtp(rtpTimestamp: number): number | null
  pushSenderReport(sample: FusionSessionSenderReportSample): void
  mapRtpTimestamp(rtpTimestamp: number): number | null
  getState(): FusionSessionState<TFrame>
  subscribe(type, listener): () => void
}

方法说明：

start()：启动 session；如果 autoConnect !== false，会同时连接遥测
connectTelemetry() / disconnectTelemetry()：手动控制遥测连接
attachVideo(video)：绑定视频元素到渲染器
updateVideoTime(videoTimestampMs, source?)：向 SDK 输入当前视频绝对时间
updateVideoTimeFromRtp(rtpTimestamp)：用内部 RTP 映射器把 RTP 时间喂给 session
pushSenderReport(sample)：输入 RTCP Sender Report 锚点
stop()：停止匹配、停止遥测、清空内部状态
destroy()：销毁 session 与 renderer

Session 事件

type FusionSessionEventMap = {
  statechange
  'telemetry:frames'
  'telemetry:match'
  'time:offset-update'
  'projection:update'
  error
}

常用事件：

statechange：整体状态变化
telemetry:match：最近一次视频时间与遥测匹配结果
time:offset-update：实时补偿值变化
projection:update：投影已使用的帧、投影状态、几何是否重建

Session 状态

interface FusionSessionState<TFrame = VmapTelemetryFrame> {
  started: boolean
  videoTimestampMs: number | null
  rawVideoTimestampMs: number | null
  telemetry: TelemetryStreamState<TFrame> | null
  realtimeProjection: FusionSessionRealtimeProjectionState<TFrame>
}

realtimeProjection 关注字段：

normalizedVideoTimestampMs
adaptiveOffsetMs
status
stableFrame
projectionFrame
geometryRebuildCount
geometrySkipCount

`telemetry`

`createWebSocketTelemetryTransport`

import { createWebSocketTelemetryTransport } from '@xingm/vmap-video/telemetry'

interface WebSocketTelemetryTransportOptions {
  url: string | (() => string | Promise<string>)
  protocols?: string | string[] | (() => string | string[] | undefined | Promise<string | string[] | undefined>)
  reconnect?: {
    enabled?: boolean
    initialDelayMs?: number
    maxDelayMs?: number
    maxRetries?: number
  }
  heartbeat?: {
    enabled?: boolean
    intervalMs?: number
    message?: TelemetryTransportOutgoingData | (() => TelemetryTransportOutgoingData)
  }
  binaryType?: BinaryType
  createSocket?: (options: { url: string; protocols?: string | string[] }) => WebSocket | Promise<WebSocket>
}

`createXingmTelemetryAdapter`

import { createXingmTelemetryAdapter } from '@xingm/vmap-video/telemetry'

interface XingmTelemetryAdapterOptions {
  aliases?: OsdFieldAliases
  previousState?: OsdRawMessage
  decodePayload?: TelemetryPayloadDecoder
  transformPayload?: TelemetryPayloadTransformer
}

用途：

标准化 OSD 字段
合并增量 OSD 状态
输出 VmapTelemetryFrame[]

`createTelemetryStream`

适合不使用 createFusionSession、但仍想复用 SDK 遥测接入流程的场景。

`adapters`

`createWsCryptoPayloadTransformer`

import { createWsCryptoPayloadTransformer } from '@xingm/vmap-video/adapters'

interface WsCryptoOptions {
  mode?: 'plain' | 'aes-gcm'
  keyMap?: Record<string, string>
  strict?: boolean
}

用途：

解密 AES-GCM WebSocket 遥测消息
可直接传给 createXingmTelemetryAdapter({ transformPayload })

`sync`

`matchTelemetryByVideoTime`

import { matchTelemetryByVideoTime } from '@xingm/vmap-video/sync'

interface TimeMatchOptions {
  thresholdMs?: number
  maxBufferSize?: number
}

interface TimeMatchResult<TFrame = VmapTelemetryFrame> {
  frame: TFrame | null
  index: number
  deltaMs: number | null
  status: 'matched' | 'empty' | 'out-of-threshold' | 'invalid-video-time'
}

`TelemetryTimeBuffer`

const buffer = new TelemetryTimeBuffer({
  thresholdMs: 5000,
  maxBufferSize: 3000,
})

方法：

push(nextFrames)
match(videoTimestampMs)
values()
clear()

`osd`

`createOsdNormalizer`

import { createOsdNormalizer } from '@xingm/vmap-video/osd'

interface OsdNormalizerOptions {
  previousState?: Record<string, unknown>
  aliases?: OsdFieldAliases
}

interface OsdNormalizer {
  normalize(raw: unknown): OsdNormalizeResult
  getState(): Record<string, unknown>
  reset(nextState?: Record<string, unknown>): void
}

`normalizeOsdMessage`

一次性归一化单条或单批消息；不保留跨调用状态。

`webrtc-time`

导出函数

ntpToUnixMs(ntpSeconds, ntpFraction)
mapRtpTimestampToAbsoluteMs(rtpTimestamp, anchor)
createRtpAbsoluteTimeMapper(initialAnchor?)

`RtpAbsoluteTimeMapper`

常用方法：

hasAnchor()
getAnchor()
reset()
updateAnchor(anchor)
updateAnchorFromSenderReport(sample)
mapRtpTimestamp(rtpTimestamp)

`projection`

`computeProjectionFootprint`

import { computeProjectionFootprint } from '@xingm/vmap-video/projection'

computeProjectionFootprint({
  frame,
  cornerOrdering: {
    enabled: true,
    yawStrategy: 'gimbal-first',
  },
})

参数说明：

frame：标准遥测帧
cornerOrdering.enabled：是否规范输入四角点顺序
cornerOrdering.yawStrategy：角点顺序判断采用的朝向策略

`createProjectionGrid`

createProjectionGrid({
  footprint,
  center,
  gridSize: 20,
})

`createVideoProjectionFrame`

createVideoProjectionFrame({
  frame,
  gridSize: 20,
  correction: {
    offsetEastMeters: 0,
    offsetNorthMeters: 0,
    rotationDeg: 0,
    scale: 1,
  },
  cornerOrdering: {
    enabled: true,
    yawStrategy: 'gimbal-first',
  },
})

`cesium`

`createCesiumProjectionRenderer`

这是推荐和 createFusionSession 配合使用的入口。

import { createCesiumProjectionRenderer } from '@xingm/vmap-video/cesium'

const renderer = createCesiumProjectionRenderer({
  viewer,
  video,
  gridSize: 20,
  heightMeters: 5,
  geometryMinUpdateIntervalMs: 120,
  geometryPositionEpsilonMeters: 1.5,
  cornerOrdering: {
    enabled: true,
    yawStrategy: 'gimbal-first',
  },
  show: false,
})

参数说明：

viewer：已有 Cesium.Viewer
video：视频纹理源
gridSize：投影网格细分数
heightMeters：投影面离地高度
correction：默认投影校正
cornerOrdering：角点顺序规范配置
geometryMinUpdateIntervalMs：几何最小重建间隔
geometryPositionEpsilonMeters：小于该位移时跳过几何重建
show：初始显隐

`createCesiumVideoProjectionLayer`

适合你自己控制投影更新时机时直接使用。

返回方法：

update({ frame, video?, correction? })
setVideo(video)
refreshTexture()
setVisible(visible)
destroy()

update() 返回：

interface VmapProjectionRendererUpdateResult {
  geometryRebuilt: boolean
  reason:
    | 'no-renderer'
    | 'initialized'
    | 'signature-unchanged'
    | 'throttled'
    | 'rebuild-forced'
    | 'rebuild-signature-changed'
}

`player`

`WebrtcPlayer`

Vue 3 组件，适合快速接入或调试。

`useWebrtcPlayer`

import { useWebrtcPlayer } from '@xingm/vmap-video/player'

interface UseWebrtcPlayerOptions {
  videoEl: Ref<HTMLVideoElement | null>
  url: Ref<string>
  iceServers?: Ref<RTCIceServer[]>
  autoReconnect?: Ref<boolean>
  reconnectInterval?: Ref<number>
  maxReconnectAttempts?: Ref<number>
  receiveAudio?: Ref<boolean>
  iceGatheringTimeout?: Ref<number>
  firstFrameTimeout?: Ref<number>
  onConnected?: () => void
  onTrack?: (stream: MediaStream) => void
  onFirstFrame?: () => void
  onError?: (error: Error) => void
  onClosed?: (reason: WebrtcPlayerCloseReason) => void
  onStateChange?: (state) => void
  onStatusChange?: (status: WebrtcPlayerStatus) => void
  onVideoAbsoluteTime?: (time: WebrtcVideoAbsoluteTime) => void
}

WebrtcVideoAbsoluteTime：

interface WebrtcVideoAbsoluteTime {
  timestampMs: number | null
  rtpTimestamp: number | null
  clockRate: number | null
  rtcpTimestampMs: number | null
  rtcpReportsSent: number | null
  source: string
}

最小示例

import { createFusionSession } from '@xingm/vmap-video'
import { createWsCryptoPayloadTransformer } from '@xingm/vmap-video/adapters'
import { createCesiumProjectionRenderer } from '@xingm/vmap-video/cesium'
import { createXingmTelemetryAdapter } from '@xingm/vmap-video/telemetry'

const renderer = createCesiumProjectionRenderer({
  viewer,
  video: videoEl,
  show: false,
})

const session = createFusionSession({
  telemetry: {
    transport: {
      type: 'websocket',
      url: telemetryWsUrl,
    },
    protocol: createXingmTelemetryAdapter({
      transformPayload: createWsCryptoPayloadTransformer({
        mode: 'aes-gcm',
        keyMap,
        strict: false,
      }),
    }),
    autoConnect: false,
    matchOptions: {
      maxBufferSize: 3000,
    },
  },
  renderer,
})

session.attachVideo(videoEl)
await session.start()
await session.connectTelemetry()
session.updateVideoTime(videoTimestampMs, source)

SDK 边界

SDK 不负责：

业务鉴权
业务设备模型
业务 UI
业务路由与状态管理
业务 Viewer 生命周期

SDK 负责：

视频时间输入
OSD 标准化
时间匹配
投影数据生成
Cesium 投影渲染适配

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

@xingm/vmap-video

文档导航

安装

导出入口

核心类型

推荐主入口：session

createFusionSession

telemetry

renderer

realtimeProjection

Session 实例方法

Session 事件

Session 状态

telemetry

createWebSocketTelemetryTransport

createXingmTelemetryAdapter

createTelemetryStream

adapters

createWsCryptoPayloadTransformer

sync

matchTelemetryByVideoTime

TelemetryTimeBuffer

osd

createOsdNormalizer

normalizeOsdMessage

webrtc-time

导出函数

RtpAbsoluteTimeMapper

projection

computeProjectionFootprint

createProjectionGrid

createVideoProjectionFrame

cesium

createCesiumProjectionRenderer

createCesiumVideoProjectionLayer

player

WebrtcPlayer

useWebrtcPlayer

最小示例

SDK 边界

推荐主入口：`session`

`createFusionSession`

`telemetry`

`renderer`

`realtimeProjection`

`telemetry`

`createWebSocketTelemetryTransport`

`createXingmTelemetryAdapter`

`createTelemetryStream`

`adapters`

`createWsCryptoPayloadTransformer`

`sync`

`matchTelemetryByVideoTime`

`TelemetryTimeBuffer`

`osd`

`createOsdNormalizer`

`normalizeOsdMessage`

`webrtc-time`

`RtpAbsoluteTimeMapper`

`projection`

`computeProjectionFootprint`

`createProjectionGrid`

`createVideoProjectionFrame`

`cesium`

`createCesiumProjectionRenderer`

`createCesiumVideoProjectionLayer`

`player`

`WebrtcPlayer`

`useWebrtcPlayer`