Data-Driven Dialog Authoring

Most bots today are authored using bot building platforms such as api.ai or wit.ai. These tools typically have 3 components:

  • Intent classifier: Maps user utterances into the most relevant intent, where intent signifies the goal or action that the user is trying to accomplish.
  • Entity recognizer: Extracts structured data, synonymously called context variables, parameters or entities, from the unstructured utterance. Examples of entities include names and locations.
  • Dialog manager: Maintains (updates) the state of the dialog (input/output context, known/unknown parameters); responds to the user utterance based on the intent and context; fetches information from and performs actions on enterprise systems.

Learning intents and entities

Typically, the structure of an intent is: {[input_contexts],[utterances],[parameters],[actions],[responses],[output_contexts]}, where:

  • input_contexts: {<flag1>,<flag2>,…} are strings where, each flag when set, act as an indicator for the existence of a particular input context for this intent.
  • utterances: {“string1”,”string2”,…} are alternate “User says” natural language statements that, each, represent a different variation of what the user might say to signal this intent. This utterance matching is conditional on the existence of the input_contexts. So, the same utterance might match different intents based on the input_contexts.
  • parameters: {entity-1,entity-2,…} are entities that are either embedded in the utterance, or obtained through an external [enterprise] system.
  • actions: {webhook-1,webhook-2,…} are typically “webhooks” which are calls to RESTful APIs (HTTP POST/GET) that pass/retrieve parameters.
  • responses: {“string1”,”string2”,…} are alternate natural language statements that, each, represent a different variation of what the bot says in response to the user utterance.
  • output_contexts: {<flag1>,<flag2>,…} are strings where, each flag when set, act as an indicator for the existence of a particular output context that is a result of execution of this intent.

To train the bot to classify intents based on utterances, bot authors upload (either manually or programmatically through APIs) example sentences that signify a particular intent. To train the bot to recognize entities, bot authors either upload values for entities or annotate utterances with entities, so the bot learns to extract entities from utterances. Both these are examples of the use of built-in machine learning to train the bot from the real conversations present in the historical chat log transcripts. Given enough data, it is possible to obtain robust intent classification and entity recognition from bot platforms such as api.ai. However, this is not true for dialog management; there is limited (if any) support for data-driven dialog management in these scripting based platforms.

Using historical data to author robust dialog scripts

Scripting a dialog amounts to “chaining” intents using the input and output context flags. This is the essence of authoring the dialog using the dialog manager. However, this is, at present, a laborious and error-prone process which results in a fragile bot that breaks when the user deviates from the expected script. For existing bot authoring platforms, I am not aware of a systematic way of authoring these dialogs automatically (or semi-automatically) using historical data. In other words, there is no learning involved in conversational dialog flows.

Currently, the only way to leverage historical chat log data for creating robust conversational flows is to test the authored flow using a “hold-out” dataset that has not been used in the dialog script authoring. This is akin to the standard practice in machine learning of a hold-out test dataset. This is an issue for enterprise bots that need to capture complex business process flows with nonlinear loops and forks. A data-driven approach to dialog script authoring will make it easier to leverage historical data to create robust dialog scripts using these authoring tools.

There are two artifacts used to create enterprise bots for applications such as customer/tech support: process flow scripts currently used by call/chat center agents, and the historical chat logs. The former is used extensively for script authoring; the latter is seldom used. One approach to leveraging the latter, in current platforms, is to first use real data to templatize the conversational flow scripts such that the most common flows are captured through reusable flows. Then, these scripted flows can be tested using variants of the flow that have not been directly used in the design. Attention is currently being focused on the UI/UX aspect of conversational flow design, which is a welcome development. However, the use of historical chat log data for Q/A or Q/C of bot scripts is currently a neglected area, at least in the context of script authoring in existing bot platforms.

In the long run, use of deep learning neural networks will likely render the current script authoring practices obsolete. These deep learning systems will be completely data driven. But, at present, none of the bot platforms feature deep learning algorithms that obviate the need for bot scripting. They may use deep learning for intent classification or entity recognition, but I am not aware of any publicly available standard bot platform (such as api.ai) that uses deep learning for dialog management. Till such purely data-driven approaches become available, bot authors will need to work on industry-standard approaches for dialog scripting, that moves this discipline from an ad-hoc art to more of a systematic engineering practice.

Advertisements

About VC Ramesh

Artificial Intelligence consultant. Current focus is on chatbots, more specifically, vertical conversational bots using deep learning (tensorflow).
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s