ump-plugin-webrtc

RTCPeerConnection / RTCDataChannel / RTCSessionDescription / RTCIceCandidate for UMP apps.

• iOS (native): a real DataChannel-over-PeerConnection bridge backed by the JitsiWebRTC pod (Google libwebrtc, BSD-3). SDP offer/answer, trickle ICE, DTLS/SCTP and bidirectional DataChannel bytes all run on the native engine.
• Android (native): the same DataChannel-over-PeerConnection bridge backed by the org.jitsi:webrtc AAR (same Google libwebrtc, BSD-3, org.webrtc.*). The C++ JSI layer (shared with iOS) bridges C++ → JNI → a Java helper (com.ump.app.UMPWebRTC) that owns the PeerConnectionFactory + handle maps. minSdkVersion 24 matches UMP's floor.
• H5 / browser: the browser's own classes win (non-overwriting global install), so nothing changes.
• Any platform without the native lib (incl. Harmony, or Android without the AAR): a feature-detect stub — classes construct cleanly so npm libs (simple-peer, webrtc-adapter, …) don't crash on import, but real ICE/SDP calls reject/throw NotSupportedError.

Media tracks (camera/mic via getUserMedia, addTrack/removeTrack, ontrack, getStats) and the <RTCVideoView> render sink are implemented on iOS (real RTCPeerConnectionFactory capture + RTCMTLVideoView renderer) and Android (org.webrtc Camera2Enumerator + VideoSource/AudioSource capture + SurfaceViewRenderer renderer). Harmony native is a later PR (Harmony has no maintained native WebRTC engine — see the design spec).

Install

npm install ump-plugin-webrtc

Usage

import { RTCPeerConnection } from 'ump-plugin-webrtc';

const pc = new RTCPeerConnection({ iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] });
pc.onicecandidate = (e) => { if (e.candidate) sendToPeer(e.candidate.toJSON()); };

const dc = pc.createDataChannel('chat');
dc.onopen = () => dc.send('hello');
dc.onmessage = (e) => console.log('got', e.data);

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
// …exchange SDP/ICE with the remote peer via your signaling channel…

Always feature-detect before relying on real connectivity:

const native = typeof (globalThis as any).__ump_rtc_pc_create === 'function';

Architecture (native path)

The four JS classes are thin proxies over native handles, identical in shape to the ump-plugin-websocket bridge:

• new RTCPeerConnection(config) → __ump_rtc_pc_create(configJson, callbackId) mints a pc_handle; the callbackId (via __ump_register_handler) parks a multi-shot JS sink the native side fires for every inbound event.
• Async ops (createOffer / createAnswer / setLocal|RemoteDescription / addIceCandidate) return Promises resolved through an op_id → {resolve,reject} map, settled when the native side posts op_result / op_error carrying that op_id.
• Inbound events (icecandidate, connection|ice|signaling statechange, datachannel, DataChannel open|message|close|error) ride the "webrtc" event channel (runtime.post_plugin_event("webrtc", …) + register_plugin_event_channel("webrtc", …)) and fan out to the right proxy by pc_handle / dc_handle.

Threading: libwebrtc fires its delegate callbacks on its own signaling thread; the native bridge never touches the JS engine there — every native→JS hop goes through post_plugin_event (which hops to the JS thread). See native/cpp/register_webrtc_jsi.cpp for the contract.

Verifying on device (loopback, no signaling server)

PR-1 cannot be exercised in CI (the sandbox can't run xcodebuild, and the JitsiWebRTC xcframework is added in your iOS env). Use this same-app loopback: two RTCPeerConnections in one process exchange SDP+ICE in-memory and open a DataChannel that echoes a message. Drop it into a UMP screen and watch the logs.

import { RTCPeerConnection } from 'ump-plugin-webrtc';

export async function webrtcLoopback(log = console.log) {
  const a = new RTCPeerConnection();
  const b = new RTCPeerConnection();

  // Trickle ICE both ways (no signaling server — wire them directly).
  a.onicecandidate = (e) => { if (e.candidate) b.addIceCandidate(e.candidate.toJSON()); };
  b.onicecandidate = (e) => { if (e.candidate) a.addIceCandidate(e.candidate.toJSON()); };

  // B answers whatever A's channel sends.
  b.ondatachannel = (e) => {
    const ch = e.channel;
    ch.onmessage = (m) => { log('B recv:', m.data); ch.send('echo:' + m.data); };
  };

  // A's outgoing DataChannel.
  const dc = a.createDataChannel('loopback');
  dc.onopen = () => { log('A dc open'); dc.send('ping'); };
  dc.onmessage = (m) => log('A recv:', m.data);   // expect "echo:ping"

  // Offer/answer handshake.
  const offer = await a.createOffer();
  await a.setLocalDescription(offer);
  await b.setRemoteDescription(offer.toJSON());

  const answer = await b.createAnswer();
  await b.setLocalDescription(answer);
  await a.setRemoteDescription(answer.toJSON());

  // …a few hundred ms later you should see:
  //   A dc open
  //   B recv: ping
  //   A recv: echo:ping
}

iOS build steps (your env)

1. Add the engine pod to your app's Podfile:

   pod 'JitsiWebRTC', '~> 124'

   (declared in this package's ump.native.ios.pods; the plugin's own UMPPluginWebRTC.podspec should carry s.dependency 'JitsiWebRTC', '~> 124' and s.ios.deployment_target = '12.0'.)
2. npx ump autolink — wires native/cpp/*.cpp into UMPPlugins.filelist, native/objc/*.{mm,m} (UMPWebRTC.mm, UMPRTCVideoHostFactory.m, register_rtcvideoview.m) into UMPPluginsObjC.filelist, and register_webrtc_jsi into the JSI dispatcher.
3. pod install and build the iOS app.
4. Run webrtcLoopback() on a device/Simulator and confirm the three log lines above.

If JitsiWebRTC is not present, UMPWebRTC.mm compiles to a no-op backend (__has_include(<WebRTC/WebRTC.h>) guard) and the JS layer falls back to the stub — so the app still builds, WebRTC just stays unsupported.

Media tracks + video render on iOS (your env)

getUserMedia / addTrack / ontrack / getStats and <RTCVideoView> run on the same JitsiWebRTC engine. Two extra steps vs. the DataChannel slice:

1. Info.plist usage strings (required — iOS hard-crashes the app the moment capture starts without them):

   <key>NSCameraUsageDescription</key>
   <string>Camera access is used for video calls.</string>
   <key>NSMicrophoneUsageDescription</key>
   <string>Microphone access is used for audio/video calls.</string>

   The backend does NOT pre-flight the permission prompt — AVCaptureDevice triggers the OS dialog on first capture. To gate it earlier use the permissions plugin (navigator.permissions) before getUserMedia.

2. The <RTCVideoView> host (UMPRTCVideoHostFactory.m, register_rtcvideoview.m) is autolinked by step 2 above and self-registers the "RTCVideoView" kind at +load. No manual wiring.

Capture a local camera/mic stream:

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: true });
const [videoTrack] = stream.getVideoTracks();
videoTrack.enabled = false;   // mute the camera
videoTrack.enabled = true;    // unmute

Render a track (local preview or a remote ontrack track):

import { RTCVideoView } from 'ump-plugin-webrtc';

// local preview
<RTCVideoView streamURL={stream} style={{ width: 240, height: 320 }} mirror />

// remote track (from pc.ontrack)
pc.ontrack = (e) => setRemote(e.track);
<RTCVideoView track={remote} style={{ flex: 1 }} objectFit="cover" />

The host binds its RTCMTLVideoView to the track via the flat trackHandle prop; a changed/zero handle rebinds. getStats() resolves a Map<string, any> keyed by stat id (each value a flat dict carrying { id, type, timestamp, …values }).

Verification gap: the .m/.mm against the JitsiWebRTC pod, live camera capture, on-GPU RTCMTLVideoView frames, and the exact getStats report shape can only be exercised on a real iOS device/Simulator with the pod installed — they are NOT covered by the JS unit tests or the C++-only CI build. Run the snippets above on device to confirm.

Android build steps (your env)

The same webrtcLoopback() above runs on Android. The engine ships as the org.jitsi:webrtc AAR (Google libwebrtc, org.webrtc.*).

1. Add the engine dependency to your app's build.gradle / build.gradle.kts:

   dependencies {
       implementation("org.jitsi:webrtc:124.+")
   }

   (declared in this package's ump.native.android.gradleDeps as documented intent — there is no autolink mechanism to inject a Maven coordinate into the host's Gradle dependencies, so you add this one line yourself, exactly as the iOS JitsiWebRTC pod is a manual Podfile line. minSdkVersion 24 is already UMP's floor.)
2. npx ump autolink — wires the shared native/cpp/*.cpp + native/android/webrtc_android.cpp into CMakeLists.gen.cmake, the native/android/*.java files (UMPWebRTC.java, UMPRTCVideoHost.java, UMPRTCVideoHostFactory.java, RegisterRTCVideoView.java) onto the app's Java srcDirs (with RegisterRTCVideoView touched from RegisterAll.gen.java), and register_webrtc_jsi into the JSI dispatcher.
3. Build the app and run it on an arm64 emulator (or device).
4. Run webrtcLoopback() and confirm the three log lines:

   A dc open
   B recv: ping
   A recv: echo:ping

Android media tracks + <RTCVideoView> (camera/mic + render)

getUserMedia / addTrack / ontrack / getStats and the <RTCVideoView> render sink run on the same org.webrtc engine (Google libwebrtc).

1. Declare the runtime permissions in your app's AndroidManifest.xml, and request them at runtime (Android 6+ dangerous permissions) BEFORE calling getUserMedia — the backend does NOT pre-flight the prompt:

   <uses-permission android:name="android.permission.CAMERA" />
   <uses-permission android:name="android.permission.RECORD_AUDIO" />
   <uses-feature android:name="android.hardware.camera" android:required="false" />

   An app that wants to pre-flight the grant can use the permissions plugin (navigator.permissions.query/request for 'camera' / 'microphone').
2. The <RTCVideoView> host (UMPRTCVideoHost.java + UMPRTCVideoHostFactory.java
   • RegisterRTCVideoView.java) is autolinked by step 2 above: RegisterAll.gen.java touches RegisterRTCVideoView.ensureRegistered(), which self-registers the "RTCVideoView" kind with UMPNativeViewRegistry. The C++ side (register_rtcvideoview.cpp) registers the kind with the runtime (placement Below, so the renderer shows through the Skia alpha hole — like <Video>).
3. Open local camera + render it:

   const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: true });
   const [videoTrack] = stream.getVideoTracks();
   // pass the track's native handle into the view via the <RTCVideoView> prop:
   <RTCVideoView streamURL={stream} style={{ width: 240, height: 320 }} mirror />

   The host wraps an org.webrtc.SurfaceViewRenderer (initialised with the process-shared EglBase context) and binds it to the track via VideoTrack.addSink(renderer) when the trackHandle prop is set.
4. Render a remote track from ontrack:

   pc.ontrack = (e) => setRemote(e.track);
   <RTCVideoView track={remote} style={{ flex: 1 }} objectFit="cover" />

If the org.jitsi:webrtc AAR is absent, webrtc_android.cpp still compiles (it only needs <jni.h>) and UMPWebRTC.java's factory() returns null (the org.webrtc.* classes aren't on the classpath) → pcCreate returns 0 → the JS layer falls back to the stub. The app still builds; WebRTC just stays unsupported (Android analog of the iOS __has_include guard).

Symbol-collision note: if your app already pulls another org.webrtc build, switch to the LiveKit relocated fork (io.github.webrtc-sdk:android → livekit.org.webrtc) — see the design spec risk #4. (This v1 backend imports the canonical org.webrtc.*.)

Migration from ump-native

- import { RTCPeerConnection } from 'ump-native';
+ import { RTCPeerConnection } from 'ump-plugin-webrtc';

No API changes.