Tap to initiate a next action for user requests

Patent metadata

Field	Value
Publication number	US-9575563-B1
Application number	US-201314144177-A
Country	US
Kind code	B1
Filing date	Dec 30, 2013
Priority date	Dec 30, 2013
Publication date	Feb 21, 2017
Grant date	Feb 21, 2017

Abstract

Embodiments may relate to intuitive user-interface features for a head-mountable device (HMD), in the context of a hybrid human and computer-automated response system. An illustrative method may involve a head-mountable device (HMD) that comprises a touchpad: (a) sending a speech-segment message to a hybrid response system, wherein the speech-segment message is indicative of a speech segment that is detected in audio data captured at the HMD, and wherein the speech-segment is associated with a first user-account with the hybrid response system, (b) receiving a response message that includes a response to the speech-segment message and an indication of a next action corresponding to the response to the speech-segment message, (c) displaying a card interface that includes an indication of the response, and (d) while displaying the response, detecting a singular touch gesture and responsively initiating the at least one next action.

First claim

We claim:

1. A method carried out by a head-mountable device comprising:
sending, a speech-segment message to a hybrid response system in response to detecting a first singular touch gesture on a touchpad of the head-mountable device (HMD),
wherein the HMD comprises a near-eye display, and wherein the touchpad is located on a side-arm of the HMD,
wherein the speech-segment message is indicative of a speech-segment detected in audio data captured at the HMD by at least one microphone, wherein the speech-segment is associated with a first user-account with the hybrid response system, and wherein the speech-segment message comprises an initial query;

receiving, at the HMD, a response message from the hybrid response system, wherein the response message comprises: both (a) an answer to the initial query and (b) an indication of a next action to be cued in anticipation of the initial query, wherein the next action is an anticipated actionable follow-up request related to the answer of the initial query; and
in response to receiving the response message:
displaying, on the HMD, a first card interface that includes an indication of the answer to the initial query; and
while displaying the first card interface that indicates the answer to the initial query, detecting touchpad data corresponding to a second singular touch gesture on the touchpad and responsively initiating the next action corresponding to the anticipated follow-up request originally indicated in the response message by displaying a second card interface of a cued next action.

2. The method of claim 1 , wherein the second singular touch gesture comprises a single tap on the touchpad.

3. The method of claim 1 , wherein initiating the next action comprises displaying a first card interface that indicates the information related to the response message.

4. The method of claim 1 , wherein the indication of the next action comprises an indication of a transaction that is associated with the response message.

5. The method of claim 4 , wherein the response message comprises an indication of a product or service, and wherein the transaction comprises a purchase of the product.

6. The method of claim 4 , wherein initiating the next action comprises initiating a purchase of a product or service via a second user-account that is associated with the first user-account.

7. The method of claim 4 , wherein initiating the at least one next action comprises displaying the first card interface that requests input that indicates whether or not to purchase a product or service via a second user-account associated with the first user-account.

8. The method of claim 1 , wherein the response message comprises proposed text for a message that can be sent via at least one second user-account associated with the first user-account, and wherein the indication of the next action comprises an indication to display the first card interface that indicates message-capable applications associated with the first user-account.

9. The method of claim 8 , wherein initiating the next action comprises displaying the first card interface that indicates the one or more message-capable applications associated with the first user-account.

10. The method of claim 1 , wherein the response message comprises proposed text for the message that can be sent via at least one second user-account associated with the first user-account, and wherein the indication of the next action comprises an indication to send a message that includes the proposed text via the at least one second user-account.

11. The method of claim 8 , wherein initiating the next action comprises sending, via the at least one second user-account, the message that includes the proposed text.

12. The method of claim 1 , wherein the HMD does not display any indication of the cued next action before detecting the second singular touch gesture and responsively initiating the next action.

13. A head-mountable device system comprising:
at least one microphone configured to provide an audio signal based on speech detected at the head-mountable device (HMD), wherein the HMD comprises a near-eye display and a touchpad that is located on a side-arm of the HMD; a network interface;
a control system configured to: detect a speech segment in the audio signal with the at least one microphone in response to detecting a first singular touch gesture on the touchpad, wherein the speech segment comprises an initial query;
send a speech-segment message to a hybrid human and automated response system, wherein the speech-segment message is indicative of the speech segment, and wherein the speech-segment message is associated with a first user-account with the hybrid human and automated response system;
receive, by the HMD, a response message from the hybrid response system, wherein the response message comprises: both (a) an answer to the initial query and (b) an indication of a next action to be cued in anticipation of the initial query, wherein the next action is responsive to an anticipated actionable follow-up request related to the answer to the initial query; and
in response to receipt of the response message:
display a first card interface that includes an indication of the answer to the initial query; and
while displaying the first card interface that indicates the answer to the initial query, detecting touchpad data corresponding to a second singular touch gesture on the touchpad and responsively initiate the next action corresponding to the anticipated follow-up request initially indicated in the response message by displaying a second card interface of a cued next action.

14. The system of claim 13 , wherein the singular touch gesture comprises a single tap on the touchpad.

15. The system of claim 13 , wherein the indication of the next action comprises information related to the response message, and wherein initiating the at least one next action comprises displaying the first card interface that indicates the information related to the response.

16. The system of claim 13 , wherein the indication of the next action comprises an indication of a transaction that is associated with the response message, and wherein the response message comprises an indication of a product or service, and wherein the transaction comprises a purchase of the product.

17. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device of a head-mountable device to perform functions comprising:
sending a speech-segment message to a hybrid human and automated response system, wherein the speech-segment message is indicative of a speech segment that is detected in audio data captured at least one microphone in response to detecting a first singular touch gesture on a touchpad on a head-mountable device (HMD) that comprises a near-eye display and a touchpad that is located on a side-arm of the HMD,
wherein the speech segment is associated with a first user-account with the hybrid human and automated response system, and wherein the speech-segment message comprises an initial query;
receiving, by the HMD, a response message from the hybrid response system, wherein the response message comprises: both (a) an answer to the initial query, and (b) an indication of a next action to be cued in anticipation of the initial query, wherein the next action is responsive to an anticipated actionable follow-up request related to the answer to the initial query;
in response to receipt of the response message: displaying a first card interface that includes an indication

Assignees

• Google Inc
• X Dev Llc

Inventors

• Li Chun Yat Frank
• Magana Daniel Rodriguez
• Teixeira Thiago
• Chen Charles
• Agarawala Anand

Classifications

• G10L15/00

  Speech recognition (G10L17/00 takes precedence) · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/017 Primary

  Head mounted · CPC title

• G06F3/044

  by capacitive means · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title