Dynamically configuring a warm word button with assistant commands

Patent metadata

Field	Value
Publication number	US-12542136-B2
Application number	US-202117532315-A
Country	US
Kind code	B2
Filing date	Nov 22, 2021
Priority date	Aug 25, 2021
Publication date	Feb 3, 2026
Grant date	Feb 3, 2026

Abstract

Implementations described herein relate to configuring a dynamic warm word button, that is associated with a client device, with particular assistant commands based on detected occurrences of warm word activation events at the client device. In response to detecting an occurrence of a given warm word activation event at the client device, implementations can determine whether user verification is required for a user that actuated the warm word button. Further, in response to determining that the user verification is required for the user that actuated the warm word button, the user verification can be performed. Moreover, in response to determining that the user that actuated the warm word button has been verified, implementations can cause an automated assistant to perform the particular assistant command associated with the warm word activation event. Audio-based and/or non-audio-based techniques can be utilized to perform the user verification.

First claim

What is claimed is:

1 . A method implemented by one or more processors of a client device, the method comprising:
detecting, based on processing one or more signals generated by the client device, an occurrence of a warm word activation event, wherein the warm word activation event is one of multiple disparate warm word activation events, wherein the warm word activation event corresponds to an incoming electronic communication that is received at the client device, and wherein the incoming electronic communication that is received at the client device is one of: a text message that is received at the client device, an email that is received at the client device, or a notification that is received at the client device; in response to detecting the occurrence of the incoming electronic communication that is received at the client device:
determining whether the incoming electronic communication that is received at the client device is the text message, the email, or the notification;
determining, based on whether the incoming electronic communication that is received at the client device is the text message, the email, or the notification, a threshold duration of time to configure a warm word button, that is associated with the client device, with a particular assistant command that, when actuated, initiates a reply electronic communication that is a reply to the incoming electronic communication that is received at the client device, wherein the threshold duration of time differs based on whether the incoming electronic communication that is received at the client device is the text message, the email, or the notification; and
configuring, for the threshold duration of time, the warm word button with the particular assistant command that, when actuated, initiates the reply electronic communication that is the reply to the incoming electronic communication that is received at the client device;
determining whether an actuation of the warm word button is detected during the occurrence of the warm word activation event or within the threshold duration of time; in response to detecting the actuation of the warm word button during the occurrence of the warm word activation event or within the threshold duration of time:
determining whether user verification is required for a user that actuated the warm word button;
determining, based on the warm word activation event corresponding to the incoming electronic communication being received at the client device, that the user verification is required for the user that actuated the warm word button;
causing the user verification for the user that actuated the warm word button to be performed;
in response to determining that the user that actuated the warm word button has been verified:
causing an automated assistant to initiate the reply electronic communication that is the reply to the incoming electronic communication that is received at the client device; and

in response to determining that the user that actuated the warm word button has not been verified:
refraining from causing an automated assistant to initiate the reply electronic communication that is the reply to the incoming electronic communication that is received at the client device; and

in response to detecting no actuation of the warm word button during the occurrence of the warm word activation event or within the threshold duration of time:
reconfiguring the warm word button with a default functionality.

2 . The method of claim 1 , determining whether the user verification is required for the user that actuated the warm word button is further based on a duration of time since a given user was last verified at the client device.

3 . The method of claim 2 , wherein determining whether the user verification is required for the user that actuated the warm word button is further based on a proximity of the given user to the client device during the duration of time since the given user was last verified at the client device.

4 . The method of claim 1 , wherein causing the user verification for the user that actuated the warm word button to be performed comprises:
determining whether to perform audio-based verification or non-audio-based verification.

5 . The method of claim 4 , further comprising:
determining to perform the audio-based verification based on voice activity of the user being detected at the same time as the actuation of the warm word button or within a threshold duration of time of actuation of the warm word button.

6 . The method of claim 5 , further comprising:
in response to determining to perform the audio-based verification:
processing, using a speaker identification (SID) model, audio data that captures the voice activity to determine whether the audio data is sufficient to verify the user that actuated the warm word button.

7 . The method of claim 6 , wherein processing, using the SID model, the audio data that captures the voice activity to determine whether the audio data is sufficient to verify the user that actuated the warm word button comprises:
processing, using a text dependent (TD) SID model as the SID model, the audio data to generate a TD speaker embedding, and wherein determining whether the audio data is sufficient to verify the user that actuated the warm word button comprises:
comparing, in an embedding space, the TD speaker embedding to one or more stored TD speaker embeddings stored locally at the client device; and
determining, based on comparing the TD speaker embedding to the one or more stored TD speaker embeddings, whether the audio data is sufficient to verify the user that actuated the warm word button.

8 . The method of claim 6 , wherein processing, using the SID model, the audio data that captures the voice activity to determine whether the audio data is sufficient to verify the user that actuated the warm word button comprises:
in response to determining that the spoken utterance is a sufficient length to perform text independent (TI) SID:
processing, using a TI SID model as the SID model, the audio data to generate a TI speaker embedding, and
wherein determining whether the audio data is sufficient to verify the user that actuated the warm word button comprises:
comparing, in an embedding space, the TI speaker embedding to one or more stored TI speaker embeddings stored locally at the client device; and
determining, based on comparing the TI speaker embedding to the one or more stored TI speaker embeddings, whether the audio data is sufficient to verify the user that actuated the warm word button.

9 . The method of claim 4 , further comprising:
determining to perform the audio-based verification based on whether the incoming electronic communication that is received at the client device is the text message, the email, or the notification.

10 . The method of claim 4 , wherein the non-audio-based verification comprises one or more of: facial verification, fingerprint verification, or password verification.

11 . The method of claim 1 , wherein the warm word button is one of: a hardware button of the client device, an agnostic software button rendered at a display of the client device, or an external hardware button that is separate from the client device.

12 . The method of claim 1 , wherein the incoming electronic communication that is received at the client device is the text message or the email.

13 . A client device comprising:
a warm word button; at least one processor; and memory storing instructions that, when executed, cause the at least one processor to:
detect, based on processing one or more signals generated by the client

Assignees

• Google Llc

Inventors

• Carbune Victor
• Gaetani Antonio
• Van Eeckhoudt Bastiaan
• Valcarce Daniel
• Golikov Michael
• Lu Justin
• Skopek Ondrej
• D'Ercole Nicolo
• Sabur Zaheed
• Behzadi Behshad
• Kothari Luv

Classifications

• G10L17/22 Primary

  Interactive procedures; Man-machine interfaces · CPC title

• H04W12/63

  Location-dependent; Proximity-dependent · CPC title

• H04W12/61

  Time-dependent · CPC title

• H04L63/0861

  using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231) · CPC title

• H04L63/083

  using passwords (cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226) · CPC title