@techsee/techsee-media-service

General Info
This package is a starting point for creating Techsee Media SDK. It contains basic framework for creating applications that utilizes media abilities. It can be video and/or audio, with or without streaming over WebRTC protocol.
The framework currently supports: Video+Voip over Techsee Video (Turn) + IE Plugin and OpenTOK + IE Plugin.

You can see a very simple demo, which demonstrates how to create very basic MediaService implementation and use it for rendering video in react component. Clone the repository, run npm i, then run npm run demo. The source code of the demo, can be found in ./demo folder.

Some basic framework principles are described below, which will help you to work with media framework.

Framework actors
MediaStreamTrack - WebRTC object, the most basic media unit.
It can be video track (WebCam or ScreenShare), or audio track (Microphone stream).

MediaStream - WebRTC object that just organizes multiple MediaStreamTracks into single group. The same MediaStreamTrack can belong to multiple MediaStreams. In Techsee Media Service implementation, MediaStream can not contain more than one MediaStreamTrack and it used only because HTML5 video element works with MediaStream object.

TechseeMediaStream - This is a logical wrapper for WebRTC MediaStreamTrack and manages some additional information regarding the specific MediaStreamTrack. For example this object has the type of the MediaStreamTrack (USER_VIDEO, AGENT_AUDIO...etc), has indication if track is local or remote.

MediaSubscriber - Techsee logical unit, that responsible to render MediaStreams into DOM. In simple words this object creates video element in html, and renders MediaStream into it, when stream is ready. MediaSubscriber is created for specific MediaStream, and whenever that stream created, the subscriber will start playing it. The subscriber will notify it's consumers about the state changes through onStateChanged method.

MediaSession - Techsee logical unit, that used to stream video over WebRTC protocol. This object is responsible to handle all aspect of peers connectivity, including signaling, or third party service handling, like OpenTok.
When connection with peer established, this service registers MediaStreamTracks that received from peer and publishes local streams the connected peer. Current implementation contains MediaSessionBase object, which have basic abstractions for all types of sessions. SessionTurn which handles the Techsee video sessions over TURN server. SessionOpentok which handles the session connectivity over OpenTok third party service.

RTCPeerConnection - WebRTC object that represents connection with single peer, when SessionTurn is used for session. The same peer is used to send and/or receive media information.

OTPeerConnection - Design time only Typescript abstraction, which represents the connectivity with peer over OpenTok session.

MediaPublisher - Techsee logical unit which organizes local MediaStreamTracks into single MediaStream for publishing.

MediaService - Techsee logical unit. Represents the Facade of the framework and glues all of the above together. Current implementation provides MediaServiceBase abstract object that contains all base logic and tools for displaying media locally or streaming it over WebRTC. Each application should create its concrete MediaService and fill specific application logic gaps. For example, currently exist DashboardMediaService in techsee-app and MobileMediaService in techee-mobile.

Consuming media service - High Level Flow

In order to be able to use media service, first you need to create and instance of it and initialize it. Depending on application, the constructor parameters can be different, but MediaServiceBase need to get the following in it's constructor:

let mediaService = new SomeSpecificMediaService(environment, webRtcInfo);

• environment: IMediaEnvironment - Service which will help understand in which environment the media service is currently running.
• webRtcSupportInfo: ClientWebRtcInfo - Information about WebRTC support on currently running device

When instance of specific MediaService was created, it should be initialized with mediaService.initMediaService(serviceOptions) method. See MediaServiceOptions for options details. The method returns promise that resolved when initialization is done. Initialization process, detects what media support currently running device have. It checks if video can be played (natively or using IE plugins), if device have mic and so on... After initialization promise resolved, the device information get be accessed through mediaService.deviceSupportInfo.

Note: Before promise returned by mediaService.initMediaService(serviceOptions) was not resolved, you not able to call mediaService.initLocalMediaStreams(serviceOptions), but you can create media subscribers if needed by calling mediaService.createSubscriber(subscriberParams).

If audio stream is needed during the session, mediaService.enableVoipDuringSession() method should be called before initializing local streams. After initialization was done and promise was resolved mediaService.initLocalMediaStreams(serviceOptions) can be called when video/audio streams should be created.
This method will start the local MediaStream creation process, and Camera/ScreenShare approval dialog will popup if there permission currently exists. initLocalMediaStreams method is called by controller which displays local media stream on mobile, and in MediaSession, when peerConnection is created. initLocalMediaStreams synchronizes the calls and insures there only one instance of stream exists, and both sides receives the same one.

As mentioned before, mediaService.createSubscriber(subscriberParams) can be called even before initialization, so whenever you have the div container that should hold video element, you can create the subscriber, by providing KnownMediaStream and div as container. Whenever the stream is ready, it will be played automatically by MediaSubscriber. MediaSubscriber will create video element, add it to div container and will insure it plays. MediaSubscriber has 'onStateChanged' method, which can be used for being notified on state changes of the subscriber. It will notify when it starts or stops playing the video, or when it has been muted or unmuted. In event listener you can inspect properties of the subscriber to see in which state it currently exists. In addition, MediaSubscriber has onDispose event handler, which will be called after MediaSubscriber was disposed and will never play video again, so references to it can be removed in order to avoid memory leaks.

Connecting to WebRTC Media Session - High Level Flow

After initialization promise was resolved, mediaService.connectToSession(sessionParams) can be called for connecting to Signaling server. Connection to session can be done before media streams were initialized, if your use case requires it. In case of of Techsee Live implementation, connection to session is done only when Terms and camera access approved by the user. Depending on configuration passed in initialization method, SessionTurn or SessionOpentok will be created to manage the media session peers. Whenever WebRTC peer will be connected, MediaSession instance will ask from mediaService for MediaStream that should be published to connected peer.