The Intelligent Agent Assistant: Bots in the Agent Desktop – Part 1


When we think of how bots fit into a Customer Service strategy, the typical scenario envisioned is often that of a customer-facing bot, providing customers a conversational support experience powered by automated intelligence. But even with the deployment of a customer-facing bot, most contact centers will still need to support a significant number of customer engagements with the help of a Customer Service Representative (CSR).

This post will be the first in a series in which we explore how we can leverage natural, conversational experiences powered by automated intelligence to empower the CSR, by building a contextually-aware Intelligent Agent Assistant, integrated into the Unified Service Desk agent desktop.

By providing a conversational Agent Assistant experience for the CSR that is tightly integrated into the Unified Service Desk, we can:

  • Expedite agent productivity with fast and intuitive access to insights and procedures
  • Provide a natural engagement experience
  • Proactively provide contextually-relevant information and alerts

 

The Intelligent Agent Assistant that we build will have:

  • Awareness of the context of the agent desktop and current customer interaction
  • The ability to provide insights, guidance or alerts based on the current context, both proactively and reactively
  • The ability to trigger and automate actions and activities within the desktop
  • The ability to be engaged via voice or text/typing

 

This 2-minute video shows the types of experiences we will be able to provide for agents at the conclusion of our series (with sound):

 

 

Building our Intelligent Agent Assistant

In Part 1 of our series, we will focus on:

  • Surfacing a bot within the Unified Service Desk
  • Enabling the bot to communicate with the Unified Service Desk by initiating actions within the desktop

 

Prerequisites:

The prerequisites for building and deploying our Intelligent Agent Assistant include:

 

Surfacing a Bot in USD

There are several options for incorporating a bot into the Unified Service Desk, including:

 

For our Intelligent Agent Assistant, we will configure a bot for the Direct Line channel, and use the WebChat control for the bot client, hosting the control in a USD Standard Web Application hosted control. This option allows us to leverage a pre-built bot client for our user interface, while still allowing us to programmatically communicate with the bot via the Direct Line REST API, sending and receiving event activities, as outlined in the Backchannel example on GitHub.

While we will get into more depth on the specifics of our bot code later on you can review some of the basics of building and deploying a bot using the Bot Builder SDK for .NET in some of the previous posts on this blog, and in the Bot Framework documentation.

Once you have deployed a bot and registered it with the Bot Framework, you can enable your bot for the Direct Line channel:
 

 

Ensure you capture your Bot Application ID during registration, and your Direct Line secret while enabling your bot on the channel.

Next, we will create a basic full-window HTML page that makes use of the WebChat, based on the fullwindow example on GitHub:

<!DOCTYPE html>
<!-- 
This page is built based on the GitHub Example seen here:
    https://github.com/Microsoft/BotFramework-WebChat/blob/master/samples/fullwindow/index.html
-->
<html>
<head>
    <meta charset="UTF-8" />
    <title>Bot Chat</title>
    <link href="https://unpkg.com/botframework-webchat/botchat.css" rel="stylesheet" />
    <link href="https://unpkg.com/botframework-webchat/botchat-fullwindow.css" rel="stylesheet" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
    <style>
            html, body {
                height: 100%;
                margin: 0;
                padding: 0;
                overflow: hidden;
            }
    </style>
</head>
<body>
    <div id="BotChatGoesHere"></div>
    <script src="https://unpkg.com/botframework-webchat/botchat.js"></script>
    <script>
            var params = BotChat.queryParams(location.search);

            var user = {
                id: params['userid'] || 'userid',
                name: params["username"] || 'username'
                };

            var bot = {
                id: params['botid'] || 'botid',
                name: params["botname"] || 'botname'
            };

            window['botchatDebug'] = params['debug'] && params['debug'] === "true";

            var botConnection = new BotChat.DirectLine({
                secret: params['s'],
                token: params['t'],
                domain: params['domain'],
                webSocket: params['webSocket'] && params['webSocket'] === "true" // defaults to true
            });

            BotChat.App({
                botConnection: botConnection,
                user: user,
                bot: bot
            }, document.getElementById("BotChatGoesHere"));

    </script>
</body>
</html>

 

After deploying this page to Azure Blob Storage, or an alternate web server, you should be able to access your bot like so:

https://<YourAzureSubdomain>.blob.core.windows.net/mycontainer/usdagentbot.html?s=<YourSecret>&botid=<YourBotId>&username=<Username>


(As a more secure alternative to using your Direct Line secret directly, you can use a Direct Line token, obtained by calling Direct Line's Generate Token.)

 

In the Dynamics 365 Web Client, we now create a new Hosted Control for our bot, using the Standard Web Application type. You can opt to make your control Global, or Session-specific; in our example we will make it Global:

 

 

We want to instantiate our control when USD is launched, so we create an Action Call to navigate our control to our bot URL:

 

 

We then add this Action Call to the DesktopReady event of the Global Manager control. Now when we launch USD, we should see our bot in the RightPanel:

 

 

Triggering Events and Actions in USD from our Bot

Now that our bot appears in USD, let’s see how we can enable our bot to use the event model of USD to trigger actions in the desktop, by creating a custom event on our Bot Assistant control, then calling it via a URL that is opened by a Card Action in our bot.

In this basic example, we will show how we can search for contacts by name, and allow our users to initiate a session with that contact.

First, we will add a custom event to our Bot Assistant control, called ContactSessionRequested:

 

 

To this event, we can call any actions we wish to in preparation for a new session, and include an Action Call called DoRoute from Bot. Note how we are including a Replacement Parameter that references a parameter from our event:

 

 

This will trigger our Window Navigation Rule, which in turn will create our session and load our contact:

 

 

We now need to call the event from our bot. USD provides a means of calling events on hosted controls using an event moniker syntax, as outlined in the Unified Service Desk documentation on user-defined events. We can use this syntax to call the event we created in USD:

http://event/?eventname=ContactSessionRequested&contactid=<contactid>

 

We will now add code to our bot to allow the bot to call the URL above when appropriate. Making use of the code from our previous blog post that allows us to query Dynamics 365 data via the Web API, we can use the following code to use our bot dialog context, and a contact search term, to find matching contacts, and return them to our user. In this example, we are using a ThumbnailCard to present our information and CardAction objects to allow our user to select a button. When the button is selected, the URL will be opened, and our custom event will be called:

public async Task SearchContactAndRespondCarousel(IDialogContext context, string keyword)
{
    // We will use a basic 'contains' filter on the contact name
    HttpResponseMessage contactResponse = await Utilities.CRMWebAPIRequest("api/data/v8.2/contacts?" +
        "$select=fullname,emailaddress1,telephone1,contactid,_parentcustomerid_value,entityimage_url&" +
        "$expand=parentcustomerid_account($select=name)&$orderby=fullname desc&$top=3&$filter=" +
        "contains(fullname, '" + keyword + "')", null, "retrieve");

    // Check to make sure our request was successful:
    if (contactResponse.IsSuccessStatusCode)
    {
        // Read our response into a JSON Array:
        string myString = contactResponse.Content.ReadAsStringAsync().Result;
        JObject contactResults =
            JObject.Parse(contactResponse.Content.ReadAsStringAsync().Result);
        JArray items = (JArray)contactResults["value"];

        // Ensure we have some results:
        if (items.Count > 0)
        {
            Dictionary<string, JObject> ContactList = new Dictionary<string, JObject>();

            IMessageActivity msgContactReply = context.MakeMessage();
            msgContactReply.Type = "message";
            msgContactReply.Attachments = new List<Attachment>();
            msgContactReply.AttachmentLayout = AttachmentLayoutTypes.Carousel;

            for (int i = 0; i < items.Count; i++)
            {

                List<CardImage> cardImages = new List<CardImage>();
                if ((string)items[i]["entityimage_url"] != null)
                {
                    cardImages.Add(new CardImage(url: WebConfigurationManager.AppSettings["dynamicsUri"] + (string)items[i]["entityimage_url"]));
                }
                else
                {
                    cardImages.Add(new CardImage(url: "https://<mygenericcontactphoto>.png"));
                }

                List<CardAction> cardButtons = new List<CardAction>();

                CardAction CreateSessionButton = new CardAction()
                {
                    Title = "Start Contact Session",
                    Type = "openUrl",
                    Value = "http://event/?eventname=ContactSessionRequested&contactid=" + (string)items[i]["contactid"]
                };
                cardButtons.Add(CreateSessionButton);

                ThumbnailCard plCard = new ThumbnailCard()
                {
                    Title = (string)items[i]["fullname"],
                    Text = "Company: " + (string)items[i]["parentcustomerid_account"]["name"],
                    Images = cardImages,
                    Buttons = cardButtons
                };

                Attachment plAttachment = plCard.ToAttachment();
                msgContactReply.Attachments.Add(plAttachment);

            }

            await context.PostAsync("Here's what I found:");
            await context.PostAsync(msgContactReply);
            await ReadyForNextCommand(context);
        }
        else
        {
            await ReadyForNextCommand(context, "I couldn't find a matching contact.");
        }

    }
    else
    {
        await ReadyForNextCommand(context, "There was an error searching.");
    }
}

public async Task ReadyForNextCommand(IDialogContext context, string message)
{
    await context.PostAsync(message);
    context.Wait(MessageReceived);
}

public async Task ReadyForNextCommand(IDialogContext context)
{
    await context.PostAsync("How else can I help you?");
    context.Wait(MessageReceived);
}

 

Included in the code is an overloaded ReadyForNextCommand method for reusability.

Here is an example of how you can call this method in the context of a dialog that uses LUIS for natural language interaction:

[LuisIntent("SearchContact")]
public async Task SearchContactAsync(IDialogContext context, LuisResult result)
{

    // retrieve the contact query terms provided by the user:
    EntityRecommendation phrase;
    if (!result.TryFindEntity("Communication.ContactName", out phrase))
    {
        await context.PostAsync("I didn't understand which customer you were searching for. Please try again");
        context.Wait(MessageReceived);
    }

    string keyword = phrase.Entity;
    await SearchContactAndRespondCarousel(context, keyword);
}

 

(see Enable language understanding with LUIS for more information on LUIS dialogs)

 

When our Card Action button that is presented to the user is clicked, the event moniker URL will be called, and a session with the selected contact will be initiated:

 

 

Stay tuned for more posts in this series, in which we will cover:

  • Making our bot aware of contextual information from USD
  • Enabling our bot to proactively notify our agent of alerts and insights based on the USD context
  • Enabling our bot for engagement via speech, in addition to typing
  • and more
Comments (0)

Skip to main content