Selectively generating expanded responses that guide continuance of a human-to-computer dialog

Patent metadata

Field	Value
Publication number	US-11238864-B2
Application number	US-201916621376-A
Country	US
Kind code	B2
Filing date	May 30, 2019
Priority date	Jun 3, 2018
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

Generating expanded responses that guide continuance of a human-to computer dialog that is facilitated by a client device and that is between at least one user and an automated assistant. The expanded responses are generated by the automated assistant in response to user interface input provided by the user via the client device, and are caused to be rendered to the user via the client device, as a response, by the automated assistant, to the user interface input of the user. An expanded response is generated based on at least one entity of interest determined based on the user interface input, and is generated to incorporate content related to one or more additional entities that are related to the entity of interest, but that are not explicitly referenced by the user interface input.

First claim

What is claimed is:

1. A method implemented by one or more processors, the method comprising:
determining at least one entity of interest, wherein determining the at least one entity of interest is based on user interface input of a user, the user interface input provided via a client device during a human-to-computer dialog session that is facilitated by the client device and that is between the user and an automated assistant;
transmitting, to each of a plurality of disparate content agents, a corresponding request that identifies the at least one entity of interest;
receiving, from each of the disparate content agents, at least one corresponding textual snippet that is responsive to the corresponding request;
processing the corresponding textual snippets, using a trained machine learning model, to determine a subgroup of the corresponding textual snippets, the subgroup of the corresponding textual snippets including:
a first textual snippet from a first content agent of the disparate content agents, and
a second textual snippet from a second content agent of the disparate content agents,

and the subgroup of the corresponding textual snippets excluding:
a third textual snippet, the third textual snippet being from the first content agent, the second content agent, or a third content agent of the disparate content agents;

combining the corresponding textual snippets of the subgroup into a composite response, wherein combining the corresponding textual snippets of the subgroup into the composite response comprises:
applying tokens of the textual snippets of the subgroup as input to a trained generative model, the trained generative model being a sequence-to-sequence model, and
generating the composite response over the trained generative model based on the input, wherein the composite response differs from the textual snippets of the subgroup and wherein generating the composite response comprises producing the composite response based on learned parameters of the trained generative model; and

causing the client device to render the composite response, the composite response rendered as a response, from the automated assistant, that is responsive to the user interface input provided during the human-to-computer dialog session.

2. The method of claim 1 , wherein processing the corresponding textual snippets, using the trained machine learning model, to determine the subgroup of the corresponding textual snippets comprises:
during a first iteration:
processing at least the first textual snippet and the second textual snippet using the trained machine learning model, and
selecting, based on the processing during the first iteration, the first textual snippet for inclusion in the subgroup, the first textual snippet selected for inclusion during the first iteration in lieu of the second textual snippet and in lieu of any additional textual snippet processed during the first iteration.

3. The method of claim 2 , wherein processing the corresponding textual snippets, using the trained machine learning model, to determine the subgroup of the corresponding textual snippets further comprises:
during a second iteration that immediately follows the first iteration:
processing at least the second textual snippet using the trained machine learning model, and
selecting, based on the processing during the second iteration, the second textual snippet for inclusion in the subgroup, the second textual snippet selected for inclusion during the second iteration in lieu of any additional textual snippet processed during the second iteration.

4. The method of claim 3 , wherein the third textual snippet is processed, using the trained machine learning model, during one or both of: the first iteration and the second iteration.

5. The method of claim 3 , wherein processing the corresponding textual snippets, using the trained machine learning model, to determine the subgroup of the corresponding textual snippets further comprises:
during a subsequent iteration that is subsequent to the first iteration and that is subsequent to the second iteration:
processing at least the third textual snippet using the trained machine learning model, and
determining, based on the processing during the subsequent iteration, to not include, in the subgroup, the third textual snippet or any additional textual snippet processed during the subsequent iteration.

6. The method of claim 3 , wherein combining the corresponding textual snippets of the subgroup into the composite response comprises:
including content of the first textual snippet in an initial portion of the composite response based on the first textual snippet being selected during the first iteration; and
including content of the second textual snippet in a next portion of the composite response, that immediately follows the initial portion, based on the second textual snippet being selected during the second iteration.

7. The method of claim 1 , wherein the trained machine learning model is a neural network model trained based on supervised training examples.

8. The method of claim 1 , wherein the trained machine learning model is a deep neural network model representing a trained policy, and the deep neural network model is trained based on rewards that are determined, during reinforcement learning, based on a reward function.

9. The method of claim 8 , wherein the reinforcement learning is based on prior human-to-computer dialog sessions of prior users, and wherein the rewards are determined based on implicit or explicit reactions of the prior users to prior composite responses, the prior composite responses including prior textual snippets selected using the deep neural network model, and the prior composite responses rendered during the prior human-to-computer dialog sessions.

10. The method of claim 1 , further comprising:
determining, based at least in part on the user interface input, an engagement measure that indicates desirability of providing the composite response in lieu of a more condensed response;
determining that the engagement measure satisfies a threshold;
wherein transmitting the corresponding request to each of the plurality of disparate content agents is responsive to determining that the engagement measure satisfies the threshold.

11. The method of claim 10 , wherein determining the engagement measure is further based on one or multiple of: a type of the client device, past interactions of the user via the client device, a time of day, a day of the week, a type of background noise, and a noise level of the background noise.

12. The method of claim 10 , wherein the user interface input is voice input and wherein determining the engagement measure is further based on one or more voice characteristics of the voice input.

13. The method of claim 1 , wherein a quantity of the textual snippets determined for inclusion in the subgroup is further based on one or multiple of: a type of the client device, past interactions of the user via the client device, a time of day, a day of the week, a type of background noise, and a noise level of the background noise.

14. The method of claim 13 , further comprising:
processing, using the trained machine learning model and along with the corresponding textual snippets, the one or multiple of: the type of the client device, the past interactions of the user via the client device, the time of day, the day of the week, the type of the background noise, and the noise level of the background noise;
wherein the processing, using the machine learning model, of the one or multiple of: the type of the client device, the past interactions o

Assignees

• Google Llc

Inventors

• Fink Michael
• Vuskovic Vladimir
• Salant Shimon Or
• Cohen Deborah
• Revach Asaf
• Kogan David
• Callahan Andrew
• Borovoy Richard
• Richardson Andrew
• Ofek Eran
• Szpektor Idan
• Berant Jonathan
• Matias Yossi

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/092

  Reinforcement learning · CPC title