Discriminating ambiguous expressions to enhance user experience

Patent metadata

Field	Value
Publication number	US-9836452-B2
Application number	US-201414586395-A
Country	US
Kind code	B2
Filing date	Dec 30, 2014
Priority date	Dec 30, 2014
Publication date	Dec 5, 2017
Grant date	Dec 5, 2017

Abstract

Methods and systems are provided for discriminating ambiguous expressions to enhance user experience. For example, a natural language expression may be received by a speech recognition component. The natural language expression may include at least one of words, terms, and phrases of text. A dialog hypothesis set from the natural language expression may be created by using contextual information. In some cases, the dialog hypothesis set has at least two dialog hypotheses. A plurality of dialog responses may be generated for the dialog hypothesis set. The dialog hypothesis set may be ranked based on an analysis of the plurality of the dialog responses. An action may be performed based on ranking the dialog hypothesis set.

First claim

What is claimed is:

1. A system comprising:
at least one processor; and
memory encoding computer executable instructions that, when executed by at least one processor, perform a method for discriminating ambiguous requests comprising:
receiving a natural language expression, wherein the natural language expression includes at least one of words, terms, and phrases of text;
creating a dialog hypothesis set from the natural language expression by using contextual information, wherein the dialog hypothesis set has a first dialog hypothesis corresponding to a first domain and a second dialog hypothesis corresponding to a second domain;
generating, from a first domain engine component and a second domain engine component, a plurality of dialog responses for the dialog hypothesis set;
ranking by machine learning techniques the first domain engine component and the second domain engine component based on an analysis of the plurality of the dialog responses; and
performing an action with the highest ranked domain engine component.

2. The system of claim 1 , wherein the natural language expression is at least one of a spoken language input and a textual input.

3. The system of claim 1 , wherein the contextual information includes at least one of information extracted from a previously received natural language expression, a response to a previously received natural language expression, client context, and knowledge content.

4. The system of claim 3 , wherein the information extracted from the previously received natural language expression includes at least a domain prediction, an intent prediction, and a slot type.

5. The system of claim 1 , wherein creating the dialog hypothesis set comprises:
extracting at least one feature from the natural language expression; and
generating at least two dialog hypotheses, where each dialog hypothesis of the dialog hypothesis set includes a different natural language expression having at least one extracted feature.

6. The system of claim 1 , wherein generating a plurality of dialog responses for the dialog hypothesis set comprises generating a plurality of responses for each dialog hypothesis of the dialog hypothesis set.

7. The system of claim 1 , wherein generating a plurality of dialog responses for the dialog hypothesis set comprises at least one of sending the dialog hypotheses to a web backend engine and sending the dialog hypotheses to a domain specific component.

8. The system of claim 1 , wherein the ranking further comprises:
extracting features from the at least two dialog hypotheses in the dialog hypothesis set; and
calculating a score for the extracted features, wherein the calculated score is indicative of the dialog hypothesis rank within the dialog hypothesis set.

9. The system of claim 1 , wherein the ranking further comprises comparing the plurality of the dialog responses with a plurality of logged dialog responses.

10. The system of claim 1 , wherein performing an action based on ranking the dialog hypothesis set comprises:
using a highest ranked dialog hypothesis to query a web backend engine for results; and
sending the results to a user of a client computing device.

11. One or more computer-readable storage media, having computer-executable instructions that, when executed by at least one processor, perform a method for training a dialog component to discriminate ambiguous requests, the method comprising:
receiving a natural language expression, wherein the natural language expression includes at least one of words, terms, and phrases of text;
creating a dialog hypothesis set from the natural language expression by using contextual information, wherein the dialog hypothesis set has a first dialog hypothesis corresponding to a first domain and a second hypothesis corresponding to a second domain;
generating, from a first domain engine component and a second domain engine component, a plurality of dialog responses for the dialog hypothesis set;
ranking by machine learning techniques the first domain engine component and the second domain engine component based on an analysis of the plurality of the dialog responses; and
performing an action with the highest ranked domain engine component.

12. The computer-readable storage media of claim 11 , wherein the method further comprises comparing the plurality of dialog responses with a plurality of logged dialog responses, wherein plurality of logged dialog responses includes a plurality of responses generated from the natural language expression.

13. The computer-readable storage media of claim 11 , wherein creating the dialog hypothesis set comprises:
extracting at least one feature from the natural language expression; and
generating at least two dialog hypotheses, where each dialog hypothesis of the dialog hypothesis set includes a different natural language expression having at least one extracted feature.

14. The computer-readable storage media of claim 12 , wherein the method further comprises:
determining whether at least one of the plurality of dialog responses matches at least one of the logged dialog responses; and
labeling at least one of the two dialog hypotheses in the dialog hypothesis set corresponding to the at least one dialog response that matches the at least one logged dialog response.

15. A computer-implemented method comprising:
receiving a natural language expression, wherein the natural language expression includes at least one of words, terms, and phrases of text;
creating a dialog hypothesis set from the natural language expression by using contextual information, wherein the dialog hypothesis set has a first dialog hypothesis corresponding to a first domain and a second dialog hypothesis corresponding to a second domain;
generating, from a first domain engine component and a second domain engine component, a plurality of dialog responses for the dialog hypothesis set;
ranking, by machine learning techniques, the first domain engine component and the second domain engine component based on an analysis of the plurality of the dialog responses; and
performing an action with the highest ranked domain engine component.

16. The computer-implemented method of claim 15 , wherein the natural language expression is at least one of a spoken language input and a textual input.

17. The computer-implemented method of claim 15 , wherein the contextual information includes at least one of information extracted from a previously received natural language expression, a response to a previously received natural language expression, client context, and knowledge content.

18. The computer-implemented method of claim 17 , wherein the information extracted from the previously received natural language expression includes at least a domain prediction, an intent prediction, and a slot type.

19. The computer-implemented method of claim 15 , wherein creating the dialog hypothesis set comprises:
extracting at least one feature from the natural language expression; and
generating at least two dialog hypotheses, where each dialog hypothesis of the dialog hypothesis set includes a different natural language expression having at least one extracted feature.

20. The computer-implemented method of claim 15 , wherein generating a plurality of dialog responses for the dialog hypothesis set comprises generating a plurality of responses for each dialog hypothesis of the dialog hypothesis set.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Robichaud Jean-Philippe
• Sarikaya Ruhi

Classifications

• G06F40/295 Primary

  Named entity recognition · CPC title

• G06F16/3325 Primary

  Reformulation based on results of preceding query · CPC title

• G06F40/35

  Discourse or dialogue representation · CPC title

• G06F16/3331

  Query processing · CPC title

• G06F16/3335 Primary

  Syntactic pre-processing, e.g. stopword elimination, stemming · CPC title