Tap to initiate a next action for user requests

Patent metadata

Field	Value
Publication number	US-9798517-B2
Application number	US-201715417609-A
Country	US
Kind code	B2
Filing date	Jan 27, 2017
Priority date	Dec 30, 2013
Publication date	Oct 24, 2017
Grant date	Oct 24, 2017

Abstract

Embodiments may relate to intuitive user-interface features for a head-mountable device (HMD), in the context of a hybrid human and computer-automated response system. An illustrative method may involve a head-mountable device (HMD) that comprises a touchpad: (a) sending a speech-segment message to a hybrid response system, wherein the speech-segment message is indicative of a speech segment that is detected in audio data captured at the HMD, and wherein the speech-segment is associated with a first user-account with the hybrid response system, (b) receiving a response message that includes a response to the speech-segment message and an indication of a next action corresponding to the response to the speech-segment message, (c) displaying a screen interface that includes an indication of the response, and (d) while displaying the response, detecting a singular touch gesture and responsively initiating the at least one next action.

First claim

We claim:

1. A method comprising:
detecting a first touch gesture on a touch interface of a client computing device, and responsively sending a speech-segment message to a hybrid response system, wherein the speech-segment message comprises an initial query corresponding to a speech segment in audio data captured at the client computing device;
receiving, at the client computing device, a response message comprising both: (a) an answer to the initial query, and (b) an indication of a next action that is cued as a response to an anticipated follow-up request related to the answer to the initial query; and
in response to receiving the response message, the client computing device:
(a) displaying, on a graphic display associated with the client computing device, a first screen indicating the answer to the initial query, and
(b) while displaying the first screen interface, detecting touchpad data corresponding to a second touch gesture on the touch interface and responsively initiating and displaying a second screen interface corresponding to the next action.

2. The method of claim 1 , wherein the second touch gesture comprises a single tap on the touch interface.

3. The method of claim 1 , wherein initiating the next action comprises displaying a first screen interface that indicates the information related to the response.

4. The method of claim 1 , wherein the indication of the next action comprises an indication of a transaction that is associated with the response message.

5. The method of claim 4 , wherein the response message comprises an indication of a product or service, and wherein the transaction comprises a purchase of the product.

6. The method of claim 4 , wherein the speech-segment message is associated with a first user-account, and wherein initiating the next action comprises initiating a purchase of a product or service via a second user-account that is associated with the first user-account.

7. The method of claim 6 , wherein initiating the at least one next action comprises displaying the first screen interface that requests input that indicates whether or not to purchase a product or service via a second user-account associated with the first user-account.

8. The method of claim 1 , wherein the speech-segment message is associated with a first user-account, and wherein the response message comprises proposed text for a message that can be sent via at least one second user-account associated with the first user-account, and wherein the indication of the next action comprises an indication to display the first screen interface that indicates message-capable applications associated with the first user-account.

9. The method of claim 8 , wherein initiating the next action comprises displaying the first screen interface that indicates the one or more message-capable applications associated with the first user-account.

10. The method of claim 1 , wherein the speech-segment message is associated with a first user-account, wherein the response message comprises proposed text for the message that can be sent via at least one second user-account associated with the first user-account, and wherein the indication of the next action comprises an indication to send a message that includes the proposed text via the at least one second user-account.

11. The method of claim 10 , wherein initiating the next action comprises sending, via the at least one second user-account, the message that includes the proposed text.

12. A computing device comprising:
at least one microphone configured to provide an audio signal based on speech detected at the computing device;
a touch interface;
a control system configured to:
detect a first touch gesture on the touch interface and responsively send a speech-segment message to a hybrid response system, wherein the speech-segment message comprises an initial query corresponding to a speech segment in audio data captured using the at least one microphone;
receiving a response message comprising both: (a) an answer to the initial query, and (b) an indication of a next action that is cued as a response to an anticipated follow-up request related to the answer to the initial query; and
in response to receipt of the response message: (a) display, on a graphic display associated with the client computing device, a first screen interface indicating the answer to the initial query, and (b) while displaying the first screen interface, detect touch data corresponding to a second touch gesture on the touch interface and responsively initiate and display a second screen interface corresponding to the next action.

13. The system of claim 12 , wherein the touch gesture comprises a single tap on the touch interface.

14. The system of claim 12 , wherein the indication of the next action comprises information related to the response, and wherein initiating the at least one next action comprises displaying a screen interface that indicates the information related to the response.

15. The system of claim 12 , wherein the indication of the next action comprises an indication of a transaction that is associated with the response, and wherein the response comprises an indication of a product or service, and wherein the transaction comprises a purchase of the product.

16. A non-transitory computer readable medium having stored therein instructions executable by at least one processor to cause the computing device of a client computing device to perform functions comprising:
detecting a first touch gesture on a touch interface of the client computing device, and responsively sending a speech-segment message to a hybrid response system, wherein the speech-segment message comprises an initial query corresponding to a speech segment in audio data captured at the client computing device;
receiving, at the client computing device, a response message comprising both: (a) an answer to the initial query, and (b) an indication of a next action that is cued as a response to an anticipated follow-up request related to the answer to the initial query; and
in response to receiving the response message, the client computing device:
(a) displaying, on a graphic display associated with the client computing device, a first screen interface indicating the answer to the initial query, and
(b) while displaying the first screen interface, detecting touchpad data corresponding to a second touch gesture on the touch interface and responsively initiating and displaying a second screen interface corresponding to the next action.

17. The non-transitory computer readable medium of claim 16 , wherein the second touch gesture comprises a single tap on the touch interface.

18. A method carried out by a hybrid response computing system comprising:
receiving a speech-segment message from a client computing device,
wherein the speech-segment message is indicative of a speech segment detected in audio data captured at the client computing device by at least one microphone, wherein the speech-segment is associated with a first user-account with the hybrid response computing system, and wherein the speech-segment message comprises an initial query;

in response to receipt of the speech segment message, and without use of any subsequent input from the user-facing computing device, the hybrid response computing system:
determining (a) an answer to the initial query, (b) an anticipated follow-up request related to the answer, and (c) a next action responsive to the anticipated follow-up request, wherein the next action is cued to be performed in response to detection of a

Assignees

• X Dev Llc

Inventors

• Li Chun Yat Frank
• Magana Daniel Rodriguez
• Teixeira Thiago
• Chen Charles
• Agarawala Anand

Classifications

• G10L15/265

  Physics · mapped topic

• G06Q30/0601

  Electronic shopping [e-shopping] · CPC title

• G06F3/03547

  Touch pads, in which fingers can move on a surface · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title