Forming chatbot output based on user state

Patent metadata

Field	Value
Publication number	US-11322143-B2
Application number	US-201916712481-A
Country	US
Kind code	B2
Filing date	Dec 12, 2019
Priority date	Sep 27, 2016
Publication date	May 3, 2022
Grant date	May 3, 2022

Abstract

Techniques are described herein for chatbots to achieve greater social grace by tracking users' states and providing corresponding dialog. In various implementations, input may be received from a user at a client device operating a chatbot, e.g., during a first session between the user and the chatbot. The input may be semantically processed to determine a state expressed by the user to the chatbot. An indication of the state expressed by the user may be stored in memory for future use by the chatbot. It may then be determined, e.g., by the chatbot based on various signals, that a second session between the user and the chatbot is underway. In various implementations, as part of the second session, the chatbot may output a statement formed from a plurality of candidate words, phrases, and/or statements based on the stored indication of the state expressed by the user.

First claim

What is claimed is:

1. A method implemented using one or more processors, the method comprising:
detecting one or more physical actions of a user, wherein detecting the one or more physical actions is during a first session, between a user and a virtual assistant, that occurs via a first client device;
analyzing, by the virtual assistant, one or more attributes of the one or more physical actions of the user to determine a state of the user during the first session;
storing, by the virtual assistant in memory and for future use by the virtual assistant, an indication of the state of the user determined during the first session;
determining, by the virtual assistant based on one or more signals, that a second session is underway, the second session being between the user and the virtual assistant, the second session occurring via a second client device, and the second session being distinct from the first session;
forming, by the virtual assistant and based on the stored indication of the state of the user, a natural language output, wherein the natural language output is associated with the state of the user determined during the first session; and
outputting, by the virtual assistant via one or more output devices of the second client device, as part of the second session, the natural language output.

2. The method of claim 1 , wherein the state of the user comprises a physical condition of the user.

3. The method of claim 1 , wherein the state of the user is a negative sentiment, and the natural language output comprises an inquiry of whether the user has improved.

4. The method of claim 1 , wherein the one or more physical actions of the user comprise a facial expression.

5. The method of claim 1 , wherein the one or more physical actions of the user comprise a gesture made by the user.

6. The method of claim 1 , wherein the one or more signals further include detection of one or more intervening interactions between the user and the first client device other than dialog between the user and the virtual assistant.

7. The method of claim 1 , wherein the one or more signals further include passage of a predetermined time interval since a last interaction between the user and the virtual assistant.

8. The method of claim 1 , wherein the one or more signals further include detection of a change in a context of the user since a last interaction between the user and the virtual assistant.

9. A system comprising:
memory storing instructions;
one or more processors that execute the instructions to:
detect, during a first session between a user and a virtual assistant that occurs via a first client device, one or more physical actions of a user;
analyze, by the virtual assistant, one or more attributes of the one or more physical actions of the user to determine a state of the user during the first session;
store, by the virtual assistant in memory and for future use by the virtual assistant, an indication of the state of the user determined during the first session;
determine, by the virtual assistant based on one or more signals, that a second session is underway, the second session being between the user and the virtual assistant and being distinct from the first session;

form, by the virtual assistant and based on the stored indication of the state of the user, a natural language output, wherein the natural language output is associated with the state of the user determined during the first session; and
output, by the virtual assistant via one or more output devices of the first client device or the second client device, as part of the second session, the natural language output.

10. The system of claim 9 , wherein the state of the user comprises a physical condition of the user.

11. The system of claim 9 , wherein the state of the user is a negative sentiment, and the natural language output comprises an inquiry of whether the user has improved.

12. The system of claim 9 , wherein the one or more physical actions of the user comprise a facial expression.

13. The system of claim 9 , wherein the one or more physical actions of the user comprise a gesture made by the user.

14. The system of claim 9 , wherein the one or more signals further include detection of one or more intervening interactions between the user and the first client device other than dialog between the user and the virtual assistant.

15. The system of claim 9 , wherein the one or more signals further include passage of a predetermined time interval since a last interaction between the user and the virtual assistant.

16. The system of claim 9 , wherein the one or more signals further include detection of a change in a context of the user since a last interaction between the user and the virtual assistant.

17. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
detecting one or more physical actions of a user, wherein detecting the one or more physical actions is during a first session, between a user and a virtual assistant, that occurs via a first client device;
analyzing, by the virtual assistant, one or more attributes of the one or more physical actions of the user to determine a state of the user during the first session;
storing, by the virtual assistant in memory and for future use by the virtual assistant, an indication of the state of the user determined during the first session;
determining, by the virtual assistant based on one or more signals, that a second session is underway, the second session being between the user and the virtual assistant, the second session occurring via a second client device, and the second session being distinct from the first session;
forming, by the virtual assistant and based on the stored indication of the state of the user, a natural language output, wherein the natural language output is associated with the state of the user determined during the first session; and
outputting, by the virtual assistant via one or more output devices of the second client device, as part of the second session, the natural language output.

Assignees

• Google Llc

Inventors

• Horling Bryan
• Kogan David
• Garrett Maryam
• Kunkle Daniel
• Quah Wan Fen Nicole
• He Ruijie
• Yuan Wangqing
• Chen Wei
• Itz Michael

Classifications

• H04L51/02

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G06F16/3329

  Natural language query formulation · CPC title