Dynamically delaying execution of automated assistant actions and/or background application requests

Patent metadata

Field	Value
Publication number	US-12073835-B2
Application number	US-202318241745-A
Country	US
Kind code	B2
Filing date	Sep 1, 2023
Priority date	Apr 26, 2019
Publication date	Aug 27, 2024
Grant date	Aug 27, 2024

Abstract

Implementations set forth herein allow a user to access a first application in a foreground of a graphical interface, and simultaneously employ an automated assistant to respond to notifications arising from a second application. The user can provide an input, such as a spoken utterance, while viewing the first application in the foreground in order to respond to notifications from the second application without performing certain intervening steps that can arise under certain circumstances. Such intervening steps can include providing a user confirmation, which can be bypassed, and/or time-limited according to a timer, which can be displayed in response to the user providing a responsive input directed at the notification. A period for the timer can be set according to one or more characteristics that are associated with the notification, the user, and/or any other information that can be associated with the user receiving the notification.

First claim

We claim:

1. A method implemented by one or more processors, the method comprising:
determining that a user provided a spoken utterance to an interface of a computing device, wherein the spoken utterance is provided while a first application and a second application are both executing at the computing device;
determining, based on the spoken utterance, that the first application is an intended target for the spoken utterance;
generating, based on processing the spoken utterance, an action for the first application to perform;
determining, based on the spoken utterance, a duration of a delay before initializing the action via an action request provided to the first application;
causing an action impending notification to be rendered, in a foreground of a graphical user interface for the duration of the delay, wherein the action-impending notification comprises:
a characterization of content of the action request, and
a dynamic graphical representation of a remaining duration of the duration of the delay;

monitoring, during the duration of the delay, for confirmation input from the user for confirming the action or modification input from the user for modifying the action; and
when, based on the monitoring, the user is determined to have provided the confirmation input for confirming the action request during the duration of the delay or is determined to have allowed the duration of the delay to expire without providing either the confirmation input or the modification input:
causing initialization of performance of the action by providing the action request to the first application,
wherein the action request is provided to the first application responsive to determining that the first application is the intended target for the spoken utterance.

2. The method of claim 1 , further comprising:
accessing contextual data characterizing one or more properties of a context in which the user provided the spoken utterance, wherein determining the duration of the delay is further based on the contextual data.

3. The method of claim 1 , wherein determining, based on the spoken utterance, the duration of the delay comprises:
generating, based on natural language content of the spoken utterance, a confidence metric; and
determining the duration of the delay based on the confidence metric.

4. The method of claim 3 , wherein generating the confidence metric is further based on input data that characterizes one or more features of one or more historical interactions of the user.

5. The method of claim 1 , wherein determining that the first application is the intended target of the spoken utterance is based on the first application having provided a most recent notification, wherein the most recent notification corresponds to an incoming message from a sender.

6. The method of claim 5 , wherein determining the duration of the delay is further based on a sender of the incoming message.

7. The method of claim 1 , wherein determining that the first application is the intended target of the spoken utterance comprises comparing content of the spoken utterance to first application data for the first application.

8. The method of claim 1 , wherein determining, based on the spoken utterance, the duration of the delay comprises:
determining the duration of the delay based on a quantity of words in the spoken utterance and/or a quantity of characters in the words.

9. The method of claim 1 , wherein the action includes sending a message, wherein natural language content of the spoken utterance includes words to include in the message, and wherein determining, based on the spoken utterance, the duration of the delay comprises:
determining the duration of the delay based on a quantity of characters in the words and/or a quantity of the words.

10. The method of claim 1 , wherein determining, based on the spoken utterance, the duration of the delay comprises:
determining one or more confidence metrics based on speech to text conversion of the spoken utterance; and
determining the duration of the delay based on the one or more confidence metrics.

11. The method of claim 1 , wherein determining, based on the spoken utterance, the duration of the delay comprises:
determining, based on historical user data, a level of similarity between the action and previous user actions; and
determining the duration of the delay based on the level of similarity.

12. A method implemented by one or more processors, the method comprising:
determining that a user provided a spoken utterance to an interface of a computing device, wherein the spoken utterance is provided while a first application and a second application are both executing at the computing device;
determining, based on the spoken utterance, that the first application is an intended target for the spoken utterance;
generating, based on processing the spoken utterance, an action for the first application to perform;
determining, based on the spoken utterance, whether to render an action impending notification in the foreground of a graphical user interface for a duration of a delay before initializing the action via an action request, wherein the action impending notification comprises:
a characterization of content of the action request, and
a dynamic graphical representation of a remaining duration of the duration of the delay;

when it is determined to render the action impending notification:
rendering the action impending notification prior to any initialization of performance of the action;

when it is determined to bypass rendering of the action impending notification
automatically initializing performance of the action by providing the action request to the first application,
wherein the action request is provided to the first application responsive to determining that the first application is the intended target for the spoken utterance.

13. The method of claim 12 , further comprising:
accessing contextual data characterizing one or more properties of a context in which the user provided the spoken utterance, wherein determining whether to render the action impending notification is further based on the contextual data.

14. The method of claim 12 , wherein determining, based on the spoken utterance, whether to render the action impending notification comprises:
generating, based on natural language content of the spoken utterance, a confidence metric; and
determining, based on the confidence metric, whether to render the action impending notification.

15. The method of claim 14 , wherein generating the confidence metric is further based on input data that characterizes one or more features of one or more historical interactions of the user.

16. The method of claim 12 , wherein determining, based on the spoken utterance, whether to render the action impending notification comprises:
determining, based on a quantity of words in the spoken utterance and/or a quantity of characters in the words, whether to render the action impending notification.

17. The method of claim 12 , wherein determining, based on the spoken utterance, whether to render the action impending notification comprises:
determining one or more confidence metrics based on speech to text conversion of the spoken utterance; and
determining, based on the one or more confidence metrics, whether to render the action impending notification.

18. The method of claim 1 , wherein determining, based on the spoken utterance, whether to render the action impending notification comprises:
determining, based on historical user

Assignees

• Google Llc

Inventors

• Burakov Denis
• Nazarov Sergey
• Behzadi Behshad
• Bertschler Mario
• Vlasyuk Bohdan
• Cotting Daniel
• Golikov Michael
• Mirelmann Lucas
• Cheng Steve
• Sabur Zaheed
• Kolak Okan
• Zhong Yan
• Ly Vinh Quoc

Classifications

• G10L2015/228

  of application context · CPC title

• G10L2015/227

  of the speaker; Human-factor methodology · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title