Phone Call Transcription and Insights with Video Indexer and Dynamics 365 (Part 2)

  • Article

This post is the second in a series in which we explore how we can leverage Microsoft’s Cognitive Services to greatly enhance the insights we can derive from phone call recordings in an automated way. We can leverage the investments Microsoft has been making in machine learning and artificial intelligence to automatically extract metadata from audio and video files, including conversation keywords, a sentiment timeline, a call transcript, and more.

In Part 2, we will build upon the Custom Actions and Workflow that we built in Part 1 by bringing in rich Insights, Transcript, and Call Audio widgets directly into our Phone Call record in Dynamics, with the ability to view our full transcript, visualize our sentiment timeline, and search the call, with an integrated media player allowing us to jump to and listen to key moments in the call:

 

Insights Widget, including interactivity with Audio Widget

 

Transcript Widget, including interactivity with Audio Widget

 

In order to embed the Video Indexer Cognitive Insights and Player widgets into our Phone Call form in Dynamics, we need to first obtain a Video Access Token from the Video Indexer API, and use that token in our widget URLs.

As part of our earlier post, we already have a Custom Workflow Activity that will retrieve a Video Access Token for us. In order to be able to access it from our Web Resource, we will create an Action that is callable from the Dynamics Web API.

 

Custom Action to Retrieve Token

Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Processes from the left navigation, and choose New.

We specify that we are creating an Action type of process that is not applicable to a specific entity:

 

 

Our Input and Output arguments for the Action will be:

  • Input
    • VideoId (String)
  • Output
    • VideoAccessToken (String)
    • VideoIndexerAccountId (String)

 

Our Action will include the following logic:

  • we call our GetVideoToken Custom Workflow Activity
  • we check to ensure that our activity completed successfully, and that we have values for our Account ID and Access Token, and if so, we assign values to our VideoAccessToken and VideoIndexerAccountId output arguments

 

 

We set the input parameters to the GetVideoToken activity like so:

 

 

After we Save and Activate our process, we are now ready to build our Web Resource that will call it.

 

Phone Call Insights Web Resource

We now create an HTML Web Resource that will be embedded in our Phone Call form in Dynamics, and will present rich and interactive Insights, Transcript and Call Audio widgets.

Our Web Resource code contains:

  • two Iframes to hold the Cognitive Insights Widget and Player Widget
  • an included Mediator JavaScript file, to manage the interactions between the two widgets
  • our included ClientGlobalContext JavaScript file to aid our Dynamics client scripting
  • JavaScript code that will execute on the onload event of our document body, and:
    • Retrieve the Video Indexer ID from our Phone Call record
    • Build and Execute a request to the Dynamics Web API to call our newly-created Action to obtain a Video Token, via the XMLHttpRequest object
    • Obtain the Video Token and Video Indexer Account ID from the response
    • Build URLs for the Insights Widget and Player widget using the account ID, video ID, and token
    • Use the built URLs to set the Source for the two widget Iframes

 

[snippet slug=phonecallinsightswebresource line_numbers=false lang=html]

 

Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Web Resources from the left navigation, and choose New, and we can now provide a name, upload our code, and Save and Publish.

 

 

Updating our Phone Call Form

We now add our newly-created Web Resource to our desired Phone Call form.

Logged in to the Dynamics 365 web client with our administrator credentials, we navigate to Settings > Customizations > Customize the System. We choose Entities > Phone Call > Forms from the left navigation, and select our Phone Call for Interactive experience form.

To the form, we add a new Tab, and within that Tab, we add our Web Resource:

 

 

We save our form, publish our customizations, and are now ready to view our Phone Call Insights. We can navigate to our Phone Call record from Part 1, and we can now open and view our interactive Insights, Transcript and Player widgets:

 
Insights Widget, including interactivity with Audio Widget

 
Transcript Widget, including interactivity with Audio Widget

 

We can now:

  • View our call sentiment timeline, and jump to any point in the call audio
  • View call keywords, and jump to any instances of those keywords in the call audio
  • View and Search our transcript, playing audio from any point in the transcript or search results
  • Translate our transcript into a variety of languages

 

By deriving call recording insights from the Video Indexer service and bringing them into Dynamics, we can now have a greater understanding of key support issues, customer sentiment, and call quality, not only for individual customers and calls, but across our organization. We can leverage this to help us optimize our support operations, and customer interactions on an ongoing basis.