Bot Framework triage between Luis and QnA Maker

  • Article

No long applicable, use the Dispatcher Service instead

I've been involved in several bot projects recently where a common pattern has emerged so I decided to produce a sample and blog about it here.

The sample is on GitHub as a C# Azure Bot Service bot: https://github.com/martinkearn/Bot-Luis-QNA-Triage

I worked with Toby Bradshaw on this and he did most the clever stuff so kudos to Toby.

Problem Space

Most customer service bots have two main requirements:

  1. Answer common questions where everyone gets the same answer.
    1. Things like "what are your opening hours", "how do I do a return", "Id like to make a complaint".
    2. The Cognitive Services QnA Maker (QnA) is ideally suited to answer these kind of FAQ-style questions
  2. Answer more vague or customer-specific questions
    1. Where the answer is either personal to the customer ("Where is my order" etc) or requires some level of natural language understanding to determine the answer ("I'd like to make an order please").
    2. The Cognitive Services Language Understanding Service (Luis) is ideally suited to understand these broader, natural language queries.

The Bot Framework SDKs have built-in patterns and classes for working with Luis and there are also plenty of samples for working with QnA. However, in the scenario above, you need to use both services and make a determination as to which one will provide the best answer whilst maintaining access to the built-in capabilities for working with Luis responses.

The Triage Solution

The solution is actually very simple and basically follows this process:

  1. Messages controller send all message to RootDialog (which is a standard IDialog) via the standard boiler-plate code of await Conversation.SendAsync(activity, () => new RootDialog());
  2. The RootDialog sends the message to both Luis and QnA and then performs several logic gates on the responses
    1. If Luis has a confidence of 70% or more (adjustable based on requirements), then send the message on to a standard LuisDialog to handle and process the intent and entities via await context.Forward(new BasicLuisDialog(), ResumeAfterSubDialog, message, CancellationToken.None);
    2. If the Luis score is less than 70% and QnA has an answer, then reply to the user with the answer from the QnA service
    3. If none of the above apply then ask the user for more details

In building the solution, we found there were a few key learnings in terms of how the Bot Framework handles dialogs.

Initially we tried to do the service requests and logic gates as part of the messages controller but we were getting problems with the context 'falling through' the dialogs and getting out of sequence. We discovered that you must always be 'in' a dialog so moved all that code into the initial RootDialog.

We experimented with both context.Forward and context.Call to find the best way to initiate the LuisDialog. We found that context.Forward was the best choice in this context because the LuisDialog needed to original message to do what it needs to do internally.

Its Open Source

The solution is open source at https://github.com/martinkearn/Bot-Luis-QNA-Triage  and I'm more than happy to accept pull requests.

Some interesting next steps are as follows:

  • A Node version
  • Find a way to avoid duplicating Luis calls between the RootDialog and LuisDialog