media-carousel

Description

The purpose of media-carousel is navigation between a gallery of video and audio files presented in a form of carousel. The media-carousel component at its very naked form has a carousel at the bottom (top) that let the user slide through the files and select one to load the player stacked at the top (bottom).

Installation and run commands

This project is only tested versus chrome 75 and firefox 67 as of now. Node version v12.3.1 and npm version 6.9.0 are used for the development.

First run

npm install

to in install all packages. Finally run

npm start

and a webpage to the media-carousel dashboard must be opened.

UPLOAD

In order to use media-carousel, all media must be saved in the Decooda S3 bucket:

export const S3Configs = {
  bucketName: "decooda-media-galleries",
  bucketRegion: "us-east-1",
  IdentityPoolId: "XYZ"
}

Note that media must be saved with the following path structure: <product_name>/<company>/<project_id>/<video_id>/filename

Each media upload must be accompanied by the following HTTP POST (X-API-KEY http header in the request; we generate one for each client app)

{
   //REQUIRED FIELDS
  s3Url, // full path including the filename
  callbackUrl:"", // POST HTTP
  modelIds:[123,456,789], // All core models used to analyze media - leave it empty or use single model id
  type:"", //video, audio
  //THE REST IS NOT REQUIRED
  date:1234, // UNIX Time stamp number
  user_id:"XYZ",
  accessToken:"", // CORE API acess token containing the model ids
  brand_id:"" // if provided the transcipt will be saved in core also
}

• Note that the video_id must be unique, otherwise you get an error in the callback.

the callbackURL will be sent the HTTP POST with

    s3Url,
    transcriptions:[
      {
      	message: "Possible retirement in overtime in price.",
      	time:12445235,
      },
      {
        message: "Possible retirement in overtime in price.",
      	time:12445235,
      }
    ],
    error:"" // if there is an error

SHOWREEL

To create a showreel the media-carousel send a POST to services.decooda.com/media/showreel (tentaive path URL - will change) with the body

{
    files:['m1','m2'], // the order is preserved in the showreel
    cuts:{
      m1:[{
        startTime: 0,
        endTime: 3
      }],
      m2:[{
        startTime: 0,
        endTime: 3
      }],
    }
    s3Url:'s3 url for showreel' // does not include the filename
    meta:{ // filtering all videos for following meta information
        emotions:[],
        valence:0, // -1 or 1
        topics:[],
        model_id:int, //
  },
  title:"showreel title",
  thumbnail:"thumbnail url",
  callbackUrl:'call back', //
  trigger:true // false to get only the processing time estimate and preview
}

The server will respond with

{
    processingTime:12, //seconds
    numVideos:2, // number of video with the hit
    s3Url:'',
    preview:{ // v2t payload subset for all hits (on trigger false)
      m1:[
        {
          //v2t payload subset for first snippet
        },
        {
          //v2t payload subset for the second snippet
        }
      ]
    }
}

if the trigger is set to true upon the completion of the task the callback url is hit by decooda process informing the from of POST request with the body

{
    s3Url:'XYZ',
    showreelId:''
}

Please note the cuts key is provided if frontend decides on the cut times for a clip. If a file does not have an associated cut key it is business as usual for the backend (5 sec padding).

More on the preview object

For the preview each file has its v2t transcription object for each snippet. For each transcription snippet at least a message before and a message after of the real hit should be included for the padding. These padded messages must have a flag pad:true . For example if file m1 has one snippet then

{
  m1: [
    {
      transcriptions: [
        {
          message: 'hello',
          time: 0,
          pad: true
        },
        {
          message: 'how is it going',
          time: 1,
          pad: false
        },
        {
          message: 'Goodbye',
          time: 2,
          pad: true
        }
      ],
      startTime: 0,
      endTime: 3
    }
  ]
}

Note that each snippet has the start and end time included.

GALLERY LOAD EVENT

When the media carousel is loaded it sends a GET HTTP request to test.v2t.decooda.com/service/media/gallery?projectId=x,company=x,productName=x,modelId=xxx and it expects (modelId is optional)

{
  files : [{
    id:'',
    src: 'url',
    status:"", // pending, complete, error
    scored:true/false,
    scoredWith:int, // model_id
    uploadDate:11111, //milliseconds - epoch
    duration:124, //duration of file in seconds
    type:"video", // or audio
    meta:{
      emotions:[],
      personas:[],
      valence:0, // -1 or 1
      topics:[]
    }
  }],
  showreels :[{
    src: 'url',
    type:"showreel"
    upload_date:11111,
    duration: 120, // duration of showreel in seconds
    meta:{
      emotions:[],
      personas:[],
      valence:0, // -1 or 1
      topics:[]
    }
  }]
}

MEDIA LOAD EVENT

The client backend must provide the transcript <product_name>/<company>/<project_id>/<video_id>/filename When a media is loaded a GET request is sent to test.v2t.decooda.com/service/media/transcript?path=<> and a detailed v2t payload...

    transcriptions:[
      {
      	message: "Possible retirement in overtime in price.",
      	time:12445235,
      },
      {
        message: "Possible retirement in overtime in price.",
      	time:12445235,
      }
    ]

Score endpoint (used for re-scoring)

{
  "urls": [String],
  "modelIds": [Int], // single int for now
  "accessToken": "",
  "callbackUrl": "" // optional
}