KeplerMediaDescriptor JS APIs

Overview

KeplerMediaDescriptor Turbo Module API provides functionality that allows apps to query the codec capabilities of the device from the platform. The API also considers the connected device's support and returns the capabilities according to the device's limitations.

Getting Started

Prerequisites

KeplerMediaDescriptor Turbo Module is dependent on the following interfaces:

1. kepler_audio_interface
2. kepler_graphics_interface

Include the necessary package dependencies

The Kepler MediaDecriptor Turbo Module is provided by the keplermediadescriptor npm package, add it as a dependency in your package.json.

  "name": "MySampleApp",
  "version": "1.0.0",

  "dependencies": {
    "@amazon-devices/keplermediadescriptor": "^1.0.0",
  },

Common Use Cases

Use this package to get the device video and audio capabilities and play your content accordingly. Some of the common use cases include:

1. Does the device support XXX video/audio codec?
2. For a given video codec, does the device support certain profile level or frame rate or bitrate or resolution?
3. Is a particular mime type supported by the device?
4. Does the connected device (TV/AVR etc. in a Fire TV stick use case), support Dolby codec?

Sample Code

Import the required interfaces from the package

import {
    KeplerMediaDescriptor,
    MediaFormat,
    VideoDecoderConfig,
    VideoFormat,
    AudioDecoderConfig,
    AudioFormat,
    MediaFormatProfileLevel,
} from "@amazon-devices/keplermediadescriptor";

Query Video Decoder capabilities.

This example shows how to query the capabilities using the codec name. It also sets the other optional attributes like resolution, bitrate, and FPS.

let mediaFormat : MediaFormat = new MediaFormat(CodecMimeType.MIME_AVC);
// 12Mbps bitrate
mediaFormat.bitrateKbps = 12000;

// 1080p Resolution
let videoFormat = new VideoFormat();
videoFormat.resolution = {
    width = 1080,
    height = 1920
};

// 60fps frame rate
videoFormat.frameRate = 60;

// use the decoder config to send all the attributes
// in a single go.
const decoderConfig = new VideoDecoderConfig();
decoderConfig.mediaFormat = mediaFormat;
decoderConfig.videoFormat = videoFormat;
decoderConfig.formatProfileLevel = profileLevel;

Note: You can also include the codec value with a mime string.

// avc codec with Main Profile and Level 4.2
let mime = "video/mp4; codecs="avc1.640028""
let profileLevel = new MediaFormatProfileLevel(mime);
decoderConfig.formatProfileLevel = profileLevel;

Once the VideoDecoderConfig object is ready, we can trigger the query API. If the capabilities array returned has at least one entry, it means that the attributes requested are supported by the device.

let videoCaps = await KeplerMediaDescriptor.queryMediaCapabilities(decoderConfig);
if (videoCaps.length > 0) {
    console.info("The device supports the requested config");
}

Read the video capabilities returned

The capabilities returned from the queryMediaCapabilities API contain further information about the device support.

// loop through the capabilities
for (let index = 0; index < videoCaps.length; index++) {
    let videoCapability = videoCaps[index];
    // check if the codec is hardware backed
    if (videoCapability.mediaCodecFeaturesCapabilities.hardwareBacked) {
        console.log("codec is hardware backed.")
    }

    // Get the min and max resolution supported by the codec:
    let resolutions = videoCapability.videoFormatCapabilities.resolutions;
    console.log("min resolution is ", resolutions[0], " max resolution is ", resolutions[1]);

    // Get the max frame rate supported by the codec for a given resolution
    let maxFps = videoCapability.videoFormatCapabilities.getMaxFrameRate(resolutions[1]);

    // Get all the color formats supported by the decoder:
    console.log(videoCapability.videoFormatCapabilities.colorFormats);

    // check if the codec supports decryption
    if (videoCapability.decoderFeaturesCapabilities.decryptionSupported) {
        console.log("decryption is supported by this codec");
    }
}

Query Audio Decoding Capabilities

Similar to the video capabilities, you can query for audio decoding support of the device.

let mediaFormat = new MediaFormat(CodecMimeType.MIME_AAC);
let audioFormat = new AudioFormat();
audioFormat.channelCount = 2;
audioFormat.sampleRate = 44100;

const decoderConfig = new AudioDecoderConfig();
decoderConfig.mediaFormat = mediaFormat;
decoderConfig.audioFormat = audioFormat;

let audioCaps = await KeplerMediaDescriptor.queryMediaCapabilities(decoderConfig);

Check Dolby support

Using the KeplerMediaDescriptor TypeScript APIs, the Dolby support of the device (in case of Fire TV stick, the connected TV, AVR, etc. are considered) can be checked.

let mediaFormat = new MediaFormat(CodecMimeType.MIME_AC3); // or MIME_EAC3
const decoderConfig = new AudioDecoderConfig();
decoderConfig.mediaFormat = mediaFormat;
let audioCaps = await KeplerMediaDescriptor.queryMediaCapabilities(decoderConfig);

if (audioCaps.length > 0) {
    console.log("the device supports DOLBY digital audio");
}

Read the audio capabilities returned

// loop through the capabilities
for (let index = 0; index < audioCaps.length; index++) {
    let audioCapability = audioCaps[index];
    if (audioCapability.decoderFeaturesCapabilities.decryptionSupported) {
        console.log("The codec supports decryption");
    }
}

FAQ

Question: Can I get all the codecs supported by the device in a single call?

Answer: use CodecMimeType.MIME_VIDEO_UNSPECIFIED and CodecMimeType.MIME_AUDIO_UNSPECIFIED as a wild card.