Generic virtual personal assistant platform

Patent metadata

Field	Value
Publication number	US-10163440-B2
Application number	US-201715399070-A
Country	US
Kind code	B2
Filing date	Jan 5, 2017
Priority date	Dec 8, 2011
Publication date	Dec 25, 2018
Grant date	Dec 25, 2018

Abstract

A method for assisting a user with one or more desired tasks is disclosed. For example, an executable, generic language understanding module and an executable, generic task reasoning module are provided for execution in the computer processing system. A set of run-time specifications is provided to the generic language understanding module and the generic task reasoning module, comprising one or more models specific to a domain. A language input is then received from a user, an intention of the user is determined with respect to one or more desired tasks, and the user is assisted with the one or more desired tasks, in accordance with the intention of the user.

First claim

What is claimed is:

1. A method for assisting a user with one or more desired tasks within a domain, the method comprising:
receiving, by a computing system comprising one or more computing devices, a verbal language input and at least one of a plurality of different kinds of non-verbal input from the user;
determining, by the computing system, from the verbal language input and the at least one of a plurality of different kinds of non-verbal input, an intention of the user with respect to the one or more desired tasks, by an executable generic language understanding module and a run-time specification comprising a model configured to a specific field of use; and
performing, by the computing system, a domain-specific task in accordance with the intention of the user, by an executable generic task reasoning module and a run-time specification comprising a task flow configured to the specific field of use.

2. The method of claim 1 , wherein the verbal language input comprises a portion of a dialog with the computer processing system having multiple rounds, and wherein the determining the intention of the user includes considering, during a given round of the dialog, a previous intention of the user determined during one or more previous rounds of the dialog.

3. The method of claim 1 , wherein the plurality of different kinds of nonverbal inputs comprises a gesture.

4. The method of claim 1 wherein the plurality of different kinds of nonverbal inputs comprises video input.

5. The method of claim 1 wherein the plurality of different kinds of nonverbal inputs comprises a click.

6. The method of claim 1 wherein the plurality of different kinds of nonverbal inputs comprises text.

7. The method of claim 1 , wherein the plurality of different kinds of nonverbal inputs comprises a multimedia input having a video portion and an audio portion.

8. The method of claim 7 , further comprising parsing the multimedia input to extract the audio portion and recognizing language in the audio portion by automatic speech recognition.

9. The method of claim 7 , further comprising parsing the multimedia input to extract the video portion and recognizing gestures in the video portion.

10. The method of claim 1 , wherein the model configured to a specific field of use utilized by the generic language understanding module includes a language model.

11. The method of claim 10 , wherein the language model comprises a grammar and an ontology.

12. The method of claim 1 , wherein the model configured to a specific field of use that are utilized by the generic task reasoning module includes a task flow model.

13. The method of claim 1 , wherein determining the intention of the user includes considering feedback from the generic task reasoning module.

14. The method of claim 1 , wherein the generic language understanding module comprises an utterance-level understanding component.

15. The method of claim 1 , wherein the generic language understanding module comprises a context-level understanding component.

16. The method of claim 1 , further comprising providing an executable speech recognition engine for execution on the computer processing system; and wherein receiving the verbal language input from the user includes receiving voice input from the user, and converting the voice input to text using the speech recognition engine.

17. The method of claim 1 , wherein the verbal language input comprises a natural language input.

18. The method of claim 1 , further comprising receiving an additional input including at least one of: a current geographical location of the user, an interaction context of the user within a web page, an image input or a non-natural language command; and wherein determining the intention of the user includes considering the additional input.

19. The method of claim 1 , wherein the desired tasks comprise at least one of: content search; content retrieval; and transaction completion.

20. The method of claim 1 , further comprising: executing the one or more desired tasks with the computer processing system.

21. The method of claim 1 , further comprising: generating the model configured to a specific field of use via a computer-assisted development tool to convert a graphical representation of at least one task flow into the set of run-time specifications.

22. A virtual personal assistant platform embodied in one or more non-transitory computer-readable media, the virtual personal assistant platform configured to cause a computing system comprising one or more computing devices to:
receive a verbal language input from the user and the at least one of a plurality of different kinds of non-verbal input;
determine, from the verbal language input and the at least one of a plurality of different kinds of non-verbal input, an intention of the user with respect to the one or more desired tasks, by an executable generic language understanding module and a run-time specification comprising a model configured to a specific field of use; and
perform, by the computing system, a domain-specific task in accordance with the intention of the user, by an executable generic task reasoning module and a run-time specification comprising a task flow configured to the specific field of use.

23. The platform of claim 22 , wherein the verbal language input of the user comprises a portion of a dialog with the computer processing system having multiple rounds, and wherein the determining the intention of the user includes considering, during a given round of the dialog, a previous intention of the user determined during one or more previous rounds of the dialog.

24. The platform of claim 22 , comprising an input module configured to input a verbal language input and at least one of a plurality of different kinds of non-verbal input.

25. The platform of claim 24 wherein the input module is further configured to convert the different kinds of non-verbal input to text.

26. The platform of claim 24 , wherein the plurality of different kinds of nonverbal input includes audio input and the input module comprises a speech recognition tool configured to recognize speech in an audio input.

27. The platform of claim 22 , comprising a sentence level understanding module configured to derive an understanding of a fragment of the verbal language input.

28. The platform of claim 27 , wherein the sentence level understanding module comprises a rule-based grammar module configured to parse the verbal language input, and a statistical parser module configured to parse the verbal language input if the rule-based grammar module fails to confidently parse the verbal language input.

29. The platform of claim 28 , comprising an interpreter module configured to analyze a user intent argument output by the sentence level understanding module and output an interpreter intent argument.

30. The platform of claim 29 , comprising a reasoner module configured to use the task flow configured to the specific field of use and the interpreter intent argument to determine a task to be performed by the computing system.

31. The platform of claim 30 , wherein (i) the reasoner module is configured to output, to the interpreter module, a reasoner intent argument indicative of the determined task to be performed, and (ii) the interpreter module is configured to use the reasoner

Assignees

• Stanford Res Inst Int

Inventors

• Yadgar Osher
• Yorke-Smith Neil
• Peintner Bart
• Tur Gokhan
• Ayan Necip Fazil
• Wolverton Michael J
• Acharya Girish
• Parimi Venkatarama Satyanarayana
• Mark William S
• Wang Wen
• Kathol Andreas
• Vincent Regis
• Franco Horacio E

Classifications

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/1822 Primary

  Parsing for meaning understanding · CPC title