Dynamic assistant suggestions during assistant browsing

Patent metadata

Field	Value
Publication number	US-12100395-B2
Application number	US-202117538736-A
Country	US
Kind code	B2
Filing date	Nov 30, 2021
Priority date	Nov 30, 2021
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

Some implementations relate to an automated assistant that can provide assistant suggestions based on content that the user has requested for the automated assistant to render at a computing device. In some instances, an assistant suggestion can correspond to a portion of the content that is not being rendered at the computing device but that is part of the content that the user is accessing. In other instances, an assistant suggestion can be based on data that can indicate how one or more users may have interacted with their automated assistant when viewing the content. Therefore, an assistant suggestion can correspond to an assistant operation that may have been initialized by another user who had previously viewed the content. In some implementations, an arrangement of the assistant suggestions can be modified according to which part of the content a user is viewing and/or user distance.

First claim

We claim:

1. A method implemented by one or more processors, the method comprising:
receiving, at a computing device, a request for an automated assistant to render content at a display interface of the computing device,
wherein the request is embodied in a spoken utterance that is provided by a user to the computing device;

causing, in response to receiving the request from the automated assistant, the display interface of the computing device to render a first portion of the content, wherein, based on finite dimensions of the display interface, the first portion of the content is rendered independent of also rendering a second portion of the content;
processing, by the automated assistant, content data that characterizes the first portion of the content rendered at the display interface of the computing device,
wherein the content data is processed in furtherance of generating a first assistant suggestion to be rendered at the display interface;

causing, based on processing the content data, the display interface of the computing device to render the first assistant suggestion with the first portion of the content;
when the first portion of the content is being rendered at the display interface:
determining, by the automated assistant, that the user has provided an input in furtherance of causing the display interface to render the second portion of the content,
wherein the second portion of the content is different than the first portion of the content, and
wherein the second portion of the content is preloaded while the first portion of the content is rendered;

processing, in response to the input, other content data that characterizes the second portion of the content,
wherein the other content data is processed in furtherance of generating a second assistant suggestion to be rendered at the display interface; and

causing, based on the other supplemental content, the display interface of the computing device to render both the second assistant suggestion and the second portion of the content,
wherein the second assistant suggestion is different than the first assistant suggestion.

2. The method of claim 1 ,
wherein the input from the user is directed to the first assistant suggestion and causes the second portion of the content to be rendered at the display interface of the computing device, and
wherein the first assistant suggestion includes natural language content that identifies the second portion of the content.

3. The method of claim 2 , wherein the input is an additional spoken utterance that is provided by the user to the automated assistant.

4. The method of claim 1 , wherein the second assistant suggestion corresponds to an operation that is performed by the automated assistant at a separate computing device that is different than the computing device.

5. The method of claim 1 , wherein processing the content data includes:
generating the first assistant suggestion based on historical interaction data,
wherein the historical interaction data characterizes a prior interaction in which the user or another user caused the computing device or another computing device to perform an action while certain content was being rendered at the computing device or the other computing device.

6. The method of claim 5 , wherein the certain content includes the first portion of the content.

7. The method of claim 1 ,
wherein the first assistant suggestion is omitted from the display interface when the second assistant suggestion is being rendered at the display interface, and
wherein the first portion of the content is omitted from the display interface when the second assistant suggestion is being rendered at the display interface.

8. The method of claim 1 ,
wherein the second assistant suggestion includes natural language content that is based on the second portion of the content, and
wherein a selection of the second assistant suggestion causes the automated assistant to initialize a separate application that is different than the automated assistant.

9. A method implemented by one or more processors, the method comprising:
receiving, at a computing device, a request for an automated assistant to render content at a display interface of the computing device,
wherein the request is embodied in a spoken utterance that is provided by a user to the computing device;

causing, in response to the request, the display interface to render the content;
processing, in response to the request, interaction data that characterizes a prior interaction in which the user caused the content to be rendered at the display interface of the computing device,
wherein a first assistant suggestion was rendered at the display interface with the content during the prior interaction;

generating, based on processing the interaction data, suggestion data that characterizes a second assistant suggestion that is different from the first assistant suggestion,
wherein the second assistant suggestion is generated further based on the content rendered at the display interface;

causing, based on the suggestion data, the display interface to render the second assistant suggestion with the content,
wherein the second assistant suggestion is selectable via a subsequent spoken utterance from the user to the automated assistant and is rendered with particular natural language content; and

causing, when the second assistant suggestion is being rendered at the display interface and based on the second assistant suggestion being rendered at the display interface, one or more speech recognition processes to be biased towards the particular natural language content of the second assistant suggestion.

10. The method of claim 9 , wherein the first assistant suggestion was rendered with particular natural language content, and the method further comprises:
causing, when the second assistant suggestion is being rendered at the display interface, one or more speech recognition processes to be biased away from the particular natural language content.

11. The method of claim 9 ,
wherein the second assistant suggestion corresponds to an operation that is initialized via the automated assistant, and
wherein the method further comprises:
causing, based on the second assistant suggestion, operation data, corresponding to the operation, to be accessed by the computing device before the user provides the subsequent spoken utterance to the automated assistant.

12. The method of claim 9 , wherein receiving the request for the automated assistant to render content at the display interface of the computing device includes:
receiving a selection of a search result, in a list of search results, being rendered at the display interface of the computing device,
wherein the second assistant suggestion is generated further based on the search result in the list of search results.

13. The method of claim 9 , wherein receiving the request for the automated assistant to render content at the display interface of the computing device includes:
receiving a selection of a search result, in a list of search results, being rendered at the display interface of the computing device,
wherein the second assistant suggestion is generated further based on one or more other search results in the list of search results.

14. A system comprising:
one or more storage devices storing instructions;
one or more processors that are operable to execute the instruction to cause the one or more processors to:
receive a request for an automated assistant to render content at a display interface of a computing device,
wherein the request is embodied in a sp

Assignees

• Google Llc

Inventors

• Cronin Stacy
• Kuang Cliff
• Wallin Abraham
• Egert-Pecot Katrina
• Klassen Sean Joseph
• Misclevitz Caleb
• PATEL Karan
• Baker Ian
• Sok Simon
• Allen Ryan

Classifications

• G10L2015/228

  of application context · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/18

  using natural language modelling · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title