Analyse

The Media plugin uses Microsoft's Video Indexer to analyze media assets for faces, sentiments, and annotations.

Media

The Media plugin uses Microsoft's Video Indexer to analyze media assets. The Video Indexer analyzes media assets for the following things:

  • Temporal Metadata: Transcripts and faces (captured in Ooyala Flex using asset metadata timelines).
  • Timeline Annotations: Topics, sentiments, audio effects, annotations, participants.
  • Asset Metadata Instance: Content moderations, audio effects, categories, and language model.

Properties:

Field Value Explanation
Icon Plugin icon
Sub-type Transcode Sub-type of this plugin
Classification Process Classification of this plugin
Deployment Internal Whether the plugin is deployed inside Ooyala Flex or within another product
Supports Configuration Whether this plugin supports configuration
Supports Scripting Whether configuration supports scripting
Scripting Context job, asset, event * What Ooyala Flex objects can be referenced from within the scripting context. * Note: the event object is only visible in the context of an action running in an event handler.
Undo Action   The action required to undo this action
Redo Action   The action required to redo this action
Auto retry   Whether to select an automatic retry
Timeout   Whether to select a timeout
Compatibility 6.2.0 What versions of Ooyala Flex this plugin works for
Lifecycle Stable Maturity of this plugin (Experimental, Unstable or Stable)

Configuration:

Field Field Type Mandatory Multiplicity Description
External Object Key string   This is an identifier that references the result of the video indexer analysis. The default for this is videoIndexer

When you upload a video to the Indexer, Microsoft provides you with an ID to reference for that upload. By default the external ID name is “video indexer”. You can change this to any name you like.
Result JSON in context boolean   When this field is set to "false", the result of media engine will be saved as standard metadata information for the asset.

When this field is set to "true", the result will be saved as a JSON in the form of job context variables.

Note: To view the variable sets, click the Variables sub-tab for the job once it has completed.
Engine Configuration complex   This is the configuration for the Media Analyser engine. This helps to analyse the media asset.
Media Analyser Engine single option   An engine which helps to analyse the media.
Note: Currently the default media analyser engine is Microsoft's Azure Video Indexer.
Key string   This is the API key for the Media Analyser engine.
Set Temporal Metadata string   Set temporal metadata for Video Indexer results. The two types of temporal metadata are:
  • Faces
  • Transcripts
To configure Faces and Transcripts, please see the Setting Up Faces Timeline and Setting Up a Transcripts Timeline sections below.
Set Timeline Annotation boolean   This field enables you to set timeline annotations for the asset. The timelines added are as follows: Topics, Sentiments, Annotation, and Audio Effects.
Note: You do not need to configure any metadata for the Topics, Sentiments, and Audio Effect timelines.

Configuring the Analysis Structure for a New Metadata Definition

The following steps are necessary in order for the analysis results to be returned in the form of a metadata instance. You must configure the metadata definition structure for the analysis, in order to set the Result in Context field to “false” in the Media plugin.

1) In the Metadata Designer, create a type (variant). See the Types section for more information.

2) Create a metadata definition and associate it with the type (variant) you created in step 1.

3) Add the mandatory Video Indexer analysis fields to your metadata definition inside the definitions array using the following PUT request:

api/metadataDefinitions/{definitionId}/definition. The fields that should be added inside the definitions array are as follows:
Note: You must replace ${DEFINITION_ID} with the definition ID of your metadata definition created in step 1.
             
                {
  "definitionId": ${DEFINITION_ID},
  "name": "root",
  "definition": [
    {
      "id": 102984,
      "name": "video-analysis",
      "displayName": "videoAnalysis",
      "description": null,
      "type": "complex",
      "searchable": false,
      "editable": false,
      "required": false,
      "formType": null,
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": [],
      "children": [
        {
          "id": 102985,
          "name": "faces",
          "displayName": "faces",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 102986,
              "name": "face-id",
              "displayName": "id",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102987,
              "name": "face-bing-id",
              "displayName": "bingId",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102988,
              "name": "face-name",
              "displayName": "name",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102989,
              "name": "face-description",
              "displayName": "description",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102990,
              "name": "face-title",
              "displayName": "title",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102991,
              "name": "creationtime",
              "displayName": "creationTime",
              "description": null,
              "type": "date",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "date",
              "format": "dd-MMM-yyyy",
              "formatDescription": "dd-MMM-yyyy",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": [],
              "timeZone": "UTC",
              "locale": "en_GB"
            },
            {
              "id": 102992,
              "name": "face-thumbnail-full-url",
              "displayName": "thumbnailFullUrl",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102993,
              "name": "face-confidence",
              "displayName": "confidence",
              "description": null,
              "type": "float",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102994,
              "name": "face-seen-duration",
              "displayName": "seenDuration",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102995,
              "name": "face-seen-duration-ratio",
              "displayName": "seenDurationRatio",
              "description": null,
              "type": "float",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102996,
              "name": "face-short-id",
              "displayName": "shortid",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        },
        {
          "id": 102997,
          "name": "participants",
          "displayName": "participants",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 102998,
              "name": "participant-id",
              "displayName": "id",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102999,
              "name": "participant-name",
              "displayName": "name",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103000,
              "name": "participant-picture-url",
              "displayName": "picture url",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        },
        {
          "id": 103001,
          "name": "content-moderations",
          "displayName": "Content Moderation",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 103002,
              "name": "content-moderation-adult-classifier-value",
              "displayName": "adultClassifierValue",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103003,
              "name": "content-moderation-racy-classifier-value",
              "displayName": "racyClassifierValue",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103004,
              "name": "content-moderation-banned-words-count",
              "displayName": "bannedWordsCount",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103005,
              "name": "content-moderation-banned-words-ratio",
              "displayName": "bannedWordsRatio",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103006,
              "name": "content-moderation-review-recommended",
              "displayName": "reviewRecommended",
              "description": null,
              "type": "boolean",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "radio",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": [],
              "options": [
                {
                  "id": 103007,
                  "name": "true",
                  "displayName": "true",
                  "default": false,
                  "value": "true"
                },
                {
                  "id": 103008,
                  "name": "false",
                  "displayName": "false",
                  "default": false,
                  "value": "false"
                }
              ]
            },
            {
              "id": 103009,
              "name": "content-moderation-is-adult",
              "displayName": "isAdult",
              "description": null,
              "type": "boolean",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "radio",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": [],
              "options": [
                {
                  "id": 103010,
                  "name": "true",
                  "displayName": "true",
                  "default": false,
                  "value": "true"
                },
                {
                  "id": 103011,
                  "name": "false",
                  "displayName": "false",
                  "default": false,
                  "value": "false"
                }
              ]
            }
          ]
        },
        {
          "id": 103012,
          "name": "audio-effects-categories",
          "displayName": "Audio Effects Categories",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 103013,
              "name": "audio-effect-category-type",
              "displayName": "type",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103014,
              "name": "audio-effect-category-key",
              "displayName": "key",
              "description": null,
              "type": "text",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "textarea",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        },
        {
          "id": 103015,
          "name": "language-models",
          "displayName": "languageModel",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 103016,
              "name": "language-model-source-language",
              "displayName": "sourceLanguage",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103017,
              "name": "language-model-language",
              "displayName": "language",
              "description": null,
              "type": "text",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "textarea",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103018,
              "name": "language-model-indexing-present",
              "displayName": "indexingPresent",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103019,
              "name": "language-model-linguistic-model-id",
              "displayName": "linguisticModelId",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        }
      ]
    }
  ],
  "rules": []
}              

Here you can see how these fields look as a metadata instance in Core:

4) Navigate to the definition in the Metadata Designer and you will see the fields.

5) Configure the Transcript and Faces Timelines. For more information, please see the Setting Up Transcript Timelines and Setting Up Faces Timelines sections below.

6) Run the Analyse Media action against a media asset.

6) When the job has completed, click the Metadata tab to view the analysis results in the form of a metadata instance.

Note: If the type (variant) is changed after you have run the Analyse Media plugin, you must run the Analyse Media again.

Configuring the Analysis Structure for an Existing Metadata Definition Structure

The following steps are necessary in order for the analysis results to be returned in the form of a metadata instance. You must configure the metadata definition structure for the analysis, in order to set the Result in Context field to “false” in the Media plugin.

1) Run the following GET request against an existing metadata definition:

api/metadataDefinitions/{definitionId}/definition

This GET request retrieves the current structure of an existing metadata definition. In the response that is returned, you will see an array field called definitions. You will need to add the mandatory Video Indexer Analysis fields inside this array.

Below is an example of a metadata definition that has been returned prior to adding the mandatory Video Indexer analysis fields. In our example the metadata definition fields are basic strings: cust1, cust2, cust 3. In step 3 we will use a PUT request to add some mandatory fields to this existing metadata definition, so that the Analyse Media action has somewhere to place the results of the analysis.

                    {  
   "definitionId":144607,
   "name":"root",
   "definition":[  
      {  
         "id":144608,
         "name":"cust1",
         "displayName":"cust1",
         "description":null,
         "type":"string",
         "searchable":false,
         "editable":true,
         "required":false,
         "formType":"text",
         "format":null,
         "formatDescription":null,
         "validation":null,
         "maxLength":-1,
         "expressionEnabled":false,
         "validationDescription":null,
         "validationHandler":null,
         "valueGeneratorType":null,
         "unitString":null,
         "commentable":false,
         "isVisible":true,
         "preProcessors":[  

         ]
      },
      {  
         "id":144609,
         "name":"cust2",
         "displayName":"cust2",
         "description":null,
         "type":"string",
         "searchable":false,
         "editable":true,
         "required":false,
         "formType":"text",
         "format":null,
         "formatDescription":null,
         "validation":null,
         "maxLength":-1,
         "expressionEnabled":false,
         "validationDescription":null,
         "validationHandler":null,
         "valueGeneratorType":null,
         "unitString":null,
         "commentable":false,
         "isVisible":true,
         "preProcessors":[  

         ]
      },
      {  
         "id":144610,
         "name":"cust3",
         "displayName":"cust3",
         "description":null,
         "type":"string",
         "searchable":false,
         "editable":true,
         "required":false,
         "formType":"text",
         "format":null,
         "formatDescription":null,
         "validation":null,
         "maxLength":-1,
         "expressionEnabled":false,
         "validationDescription":null,
         "validationHandler":null,
         "valueGeneratorType":null,
         "unitString":null,
         "commentable":false,
         "isVisible":true,
         "preProcessors":[  

         ]
      }
   ],
   "rules":[  

   ]
}
                    
                    
                    
                    
           

3) Add the mandatory Video Indexer analysis fields to your existing metadata definition inside the definition array using the following PUT request:

api/metadataDefinitions/{definitionId}/definition

The fields that should be added inside the definition array are as follows:

{
"definitionId": 144607,
"name": "root",
"definition": [
{
"id": 144608,
"name": "cust1",
"displayName": "cust1",
"description": null,
"type": "string",
"searchable": false,
"editable": true,
"required": false,
"formType": "text",
"format": null,
"formatDescription": null,
"validation": null,
"maxLength": -1,
"expressionEnabled": false,
"validationDescription": null,
"validationHandler": null,
"valueGeneratorType": null,
"unitString": null,
"commentable": false,
"isVisible": true,
"preProcessors": []
},
{
"id": 144609,
"name": "cust2",
"displayName": "cust2",
"description": null,
"type": "string",
"searchable": false,
"editable": true,
"required": false,
"formType": "text",
"format": null,
"formatDescription": null,
"validation": null,
"maxLength": -1,
"expressionEnabled": false,
"validationDescription": null,
"validationHandler": null,
"valueGeneratorType": null,
"unitString": null,
"commentable": false,
"isVisible": true,
"preProcessors": []
},
{
"id": 144610,
"name": "cust3",
"displayName": "cust3",
"description": null,
"type": "string",
"searchable": false,
"editable": true,
"required": false,
"formType": "text",
"format": null,
"formatDescription": null,
"validation": null,
"maxLength": -1,
"expressionEnabled": false,
"validationDescription": null,
"validationHandler": null,
"valueGeneratorType": null,
"unitString": null,
"commentable": false,
"isVisible": true,
"preProcessors": []
},
{
      "id": 102984,
      "name": "video-analysis",
      "displayName": "videoAnalysis",
      "description": null,
      "type": "complex",
      "searchable": false,
      "editable": false,
      "required": false,
      "formType": null,
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": [],
      "children": [
        {
          "id": 102985,
          "name": "faces",
          "displayName": "faces",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 102986,
              "name": "face-id",
              "displayName": "id",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102987,
              "name": "face-bing-id",
              "displayName": "bingId",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102988,
              "name": "face-name",
              "displayName": "name",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102989,
              "name": "face-description",
              "displayName": "description",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102990,
              "name": "face-title",
              "displayName": "title",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102991,
              "name": "creationtime",
              "displayName": "creationTime",
              "description": null,
              "type": "date",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "date",
              "format": "dd-MMM-yyyy",
              "formatDescription": "dd-MMM-yyyy",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": [],
              "timeZone": "UTC",
              "locale": "en_GB"
            },
            {
              "id": 102992,
              "name": "face-thumbnail-full-url",
              "displayName": "thumbnailFullUrl",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102993,
              "name": "face-confidence",
              "displayName": "confidence",
              "description": null,
              "type": "float",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102994,
              "name": "face-seen-duration",
              "displayName": "seenDuration",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102995,
              "name": "face-seen-duration-ratio",
              "displayName": "seenDurationRatio",
              "description": null,
              "type": "float",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102996,
              "name": "face-short-id",
              "displayName": "shortid",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        },
        {
          "id": 102997,
          "name": "participants",
          "displayName": "participants",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 102998,
              "name": "participant-id",
              "displayName": "id",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 102999,
              "name": "participant-name",
              "displayName": "name",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103000,
              "name": "participant-picture-url",
              "displayName": "picture url",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        },
        {
          "id": 103001,
          "name": "content-moderations",
          "displayName": "Content Moderation",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 103002,
              "name": "content-moderation-adult-classifier-value",
              "displayName": "adultClassifierValue",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103003,
              "name": "content-moderation-racy-classifier-value",
              "displayName": "racyClassifierValue",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103004,
              "name": "content-moderation-banned-words-count",
              "displayName": "bannedWordsCount",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103005,
              "name": "content-moderation-banned-words-ratio",
              "displayName": "bannedWordsRatio",
              "description": null,
              "type": "double",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103006,
              "name": "content-moderation-review-recommended",
              "displayName": "reviewRecommended",
              "description": null,
              "type": "boolean",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "radio",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": [],
              "options": [
                {
                  "id": 103007,
                  "name": "true",
                  "displayName": "true",
                  "default": false,
                  "value": "true"
                },
                {
                  "id": 103008,
                  "name": "false",
                  "displayName": "false",
                  "default": false,
                  "value": "false"
                }
              ]
            },
            {
              "id": 103009,
              "name": "content-moderation-is-adult",
              "displayName": "isAdult",
              "description": null,
              "type": "boolean",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "radio",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": [],
              "options": [
                {
                  "id": 103010,
                  "name": "true",
                  "displayName": "true",
                  "default": false,
                  "value": "true"
                },
                {
                  "id": 103011,
                  "name": "false",
                  "displayName": "false",
                  "default": false,
                  "value": "false"
                }
              ]
            }
          ]
        },
        {
          "id": 103012,
          "name": "audio-effects-categories",
          "displayName": "Audio Effects Categories",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 103013,
              "name": "audio-effect-category-type",
              "displayName": "type",
              "description": null,
              "type": "long",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": "#,##0.###",
              "formatDescription": "#,##0.###",
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103014,
              "name": "audio-effect-category-key",
              "displayName": "key",
              "description": null,
              "type": "text",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "textarea",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        },
        {
          "id": 103015,
          "name": "language-models",
          "displayName": "languageModel",
          "description": null,
          "type": "complex",
          "multiplicity": "0..*",
          "searchable": false,
          "editable": false,
          "required": false,
          "formType": null,
          "format": null,
          "formatDescription": null,
          "validation": null,
          "maxLength": -1,
          "expressionEnabled": false,
          "validationDescription": null,
          "validationHandler": null,
          "valueGeneratorType": null,
          "unitString": null,
          "commentable": false,
          "isVisible": true,
          "preProcessors": [],
          "children": [
            {
              "id": 103016,
              "name": "language-model-source-language",
              "displayName": "sourceLanguage",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103017,
              "name": "language-model-language",
              "displayName": "language",
              "description": null,
              "type": "text",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "textarea",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103018,
              "name": "language-model-indexing-present",
              "displayName": "indexingPresent",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            },
            {
              "id": 103019,
              "name": "language-model-linguistic-model-id",
              "displayName": "linguisticModelId",
              "description": null,
              "type": "string",
              "searchable": false,
              "editable": true,
              "required": false,
              "formType": "text",
              "format": null,
              "formatDescription": null,
              "validation": null,
              "maxLength": -1,
              "expressionEnabled": false,
              "validationDescription": null,
              "validationHandler": null,
              "valueGeneratorType": null,
              "unitString": null,
              "commentable": false,
              "isVisible": true,
              "preProcessors": []
            }
          ]
        }
      ]
    }
],
"rules": []
}   
                            
                     

Once you have run the PUT request, the fields will be added to your metadata definition.

Note: The mandatory Video Indexer analysis fields must be added to your metadata definition in order for the analysis results to be returned as a metadata instance.

4) Navigate to the definition in the Metadata Designer and you will see the newly added fields.

5) Configure the Transcript and Faces Timelines. For more information, please see the Setting Up Transcript Timelines and Setting Up Faces Timelines sections below.

6) Run the Analyse Media action against an asset.

6) When the job has completed, click the Metadata tab to view the analysis results in the form of a metadata instance.

Note: If the type (variant) is changed after you have run the Analyse Media plugin, you must run the Analyse Media again.

Setting Up a Transcript Timeline

The Microsoft Video Indexer can extract speech and convert it into text. This is represented as a track on the timeline below the player in MAM.

To configure this, follow these steps:

1) In the Metadata Designer, navigate to the variant you have created.

2) Create a timeline with the name “Transcripts”. For more information about how to create timelines, please see the Timeline Configuration guide.

3) Create a metadata definition and then associate it with the Transcript timeline. This metadata definition must have the following fields (all of which must be strings):

  • id
  • text
  • confidence
  • participentid
 {
  "definitionId": ${DEFINITION_ID},
  "name": "root",
  "definition": [
    {
      "name": "id",
      "displayName": "id",
      "description": null,
      "type": "string",
      "searchable": false,
      "editable": true,
      "required": false,
      "formType": "text",
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": []
    },
    {
      "name": "text",
      "displayName": "text",
      "description": null,
      "type": "string",
      "searchable": false,
      "editable": true,
      "required": false,
      "formType": "text",
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": []
    },
    {
      "name": "confidence",
      "displayName": "confidence",
      "description": null,
      "type": "string",
      "searchable": false,
      "editable": true,
      "required": false,
      "formType": "text",
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": []
    },
    {
      "name": "participantid",
      "displayName": "participantid",
      "description": null,
      "type": "string",
      "searchable": false,
      "editable": true,
      "required": false,
      "formType": "text",
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": []
    }
  ],
  "rules": []
}

           

Setting Up a Faces Timeline

The Video Indexer can recognise faces in a video. These might be celebrities such as actors and presenters. Each face is matched against a comprehensive celebrity database. The Video Indexer uses this database to become familiar with a particular person’s face, so that it can accurately detect them in future videos. This is represented as a track on the timeline below the player in MAM.

To configure this, follow these steps:

1) In the Metadata Designer, navigate to the variant you have created.

2) Create a timeline with the name “Faces”. For more information about how to create timelines, please see the Timeline Configuration guide.

3) You must create a metadata definition and then associate it with the Faces timeline. This metadata definition must have the following fields (all of which must be strings):
  • id
  • name
  • confidence
           
           {
  "definitionId": ${DEFINITION_ID},
  "name": "root",
  "definition": [
    {
      "name": "id",
      "displayName": "id",
      "description": null,
      "type": "long",
      "searchable": false,
      "editable": true,
      "required": false,
      "formType": "text",
      "format": "#,##0.###",
      "formatDescription": "#,##0.###",
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": []
    },
    {
      "name": "name",
      "displayName": "name",
      "description": null,
      "type": "string",
      "searchable": false,
      "editable": true,
      "required": false,
      "formType": "text",
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": []
    },
    {
      "name": "confidence",
      "displayName": "confidence",
      "description": null,
      "type": "text",
      "searchable": false,
      "editable": true,
      "required": false,
      "formType": "textarea",
      "format": null,
      "formatDescription": null,
      "validation": null,
      "maxLength": -1,
      "expressionEnabled": false,
      "validationDescription": null,
      "validationHandler": null,
      "valueGeneratorType": null,
      "unitString": null,
      "commentable": false,
      "isVisible": true,
      "preProcessors": []
    }
  ],
  "rules": []
}

           
           
           
       
https://help.ooyala.com/sites/all/libraries/dita/en/media-logistics/flex/dev/60/action_plugins_60_analyse.html

Was this article helpful?