Input determination method

Patent metadata

Field	Value
Publication number	US-9423870-B2
Application number	US-201213466673-A
Country	US
Kind code	B2
Filing date	May 8, 2012
Priority date	May 8, 2012
Publication date	Aug 23, 2016
Grant date	Aug 23, 2016

Abstract

Methods and systems for determining intent in voice and gesture interfaces are described. An example method includes determining that a gaze direction is in a direction of a gaze target, and determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target. The method may also include providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed. According to the method, a voice or gesture command that is received after the predetermined time period has elapsed may be determined to be an input for a computing system. Additional example systems and methods are described herein.

First claim

What is claimed is:

1. A method comprising:
de-activating voice control of a voice-controlled device located remotely from a user, wherein the voice-controlled device comprises a gaze tracking component and an indicator component;
determining, using the gaze tracking component of the voice-controlled device, and while the voice control is deactivated, whether a gaze of a user is in a direction of a gaze target, wherein the gaze target comprises a portion of the voice-controlled device in which the indicator component and the gaze tracking component are positioned;
in response to determining that the gaze of the user is not in the direction of the gaze target, and while the voice control is deactivated, providing, using the indicator component and at the gaze target, a first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected and that the voice control is deactivated;
subsequent to providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected, and while the voice control is deactivated, determining, using the gaze tracking component of the voice-controlled device, that the gaze of the user is in the direction of the gaze target;
responsive to the subsequent determination that the gaze of the user is in the direction of the gaze target, and while the voice control is deactivated, switching from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing, to the user using the indicator component and at the gaze target, a second human-perceivable indication that indicates that: the gaze of the user is in the direction of the gaze target, the voice control is deactivated, and maintaining the gaze in the direction of the gaze target for a predetermined time period will activate the voice control;
subsequent to switching from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing the second human-perceivable indication that indicates that the gaze of the user is in the direction of the gaze target, and while the voice control is deactivated, determining, using the gaze tracking component of the voice-controlled device, that the predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target;
responsive to determining that the predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target:
switching from providing the second human-perceivable indication to providing, to the user using the indicator component and at the gaze target, a third human-perceivable indication that indicates that the gaze of the user has been maintained in the direction of the gaze target for at least the predetermined time period and that the voice control is activated, wherein the first, second, and third human-perceivable indications are different from one another, and
switching a voice input component from an off-state to an on-state; and

determining that a voice command that is received after switching the voice input component to the on-state is an input for the voice-controlled device.

2. The method of claim 1 , further comprising:
determining, using audio localization, a direction to a source of the voice command; and
determining the voice command is an input for the voice-controlled device when the direction to the source of the voice command matches a direction to an origin of the gaze of the user.

3. The method of claim 1 , further comprising:
determining, using audio localization, a direction to a source of another voice command; and
determining the other voice command is not an input for the voice-controlled device when the direction to the source of the other voice command matches a direction to a position other than the origin of the gaze of the user.

4. The method of claim 1 , further comprising:
determining the voice command is an input for the voice-controlled device when the voice command is received within a second predetermined time period of when the third human-perceivable indication is provided.

5. The method of claim 1 , wherein the indicator component comprises a light source.

6. The method of claim 1 , wherein the indicator component comprises a speaker.

7. The method of claim 1 , wherein determining that the gaze of the user is in the direction of the gaze target comprises:
determining an expected angle from an origin of the gaze of the user to a location of the gaze target; and
determining that the gaze of the user is in the direction of the gaze target based on a comparison of the expected angle and the gaze of the user.

8. The method of claim 7 , further comprising:
determining that the gaze of the user is within a threshold of the expected angle; and
providing feedback indicating a relationship between the gaze of the user and the expected angle, wherein the feedback includes information associated with a change in the gaze of the user to align the gaze of the user with the expected angle.

9. The method of claim 1 , further comprising once provision of the second human-perceivable indication is initiated, continuously providing the second human-perceivable indication while the gaze of the user is in the direction of the gaze target until it is determined that the predetermined time period has elapsed.

10. The method of claim 1 , wherein providing the second human-perceivable indication and providing the third human-perceivable indication comprise illuminating one or more of a plurality of light-emitting diodes.

11. The method of claim 10 :
wherein providing the second human-perceivable indication comprises initiating blinking of one or more of the plurality of light-emitting diodes, and
wherein providing the third human-perceivable indication comprises providing a sustained illumination of the one or more of the plurality of light-emitting diodes.

12. The method of claim 11 , wherein the plurality of light-emitting diodes forms at least a portion of a ring around the gaze target.

13. The method of claim 10 :
wherein providing the second human-perceivable indication comprises initiating blinking of one or more of the plurality of light-emitting diodes, and
wherein providing the third human-perceivable indication comprises increasing a rate of blinking of the one or more of the plurality of light-emitting diodes.

14. The method of claim 1 , further comprising:
obtaining, by the voice-controlled device, one or more images; and
processing, by the voice-controlled device, the one or more images to determine the gaze of the user.

15. A non-transitory computer-readable medium having stored therein instructions executable by a voice-controlled device to cause the voice-controlled device to perform functions comprising:
de-activating voice control of the voice-controlled device, wherein the voice-controlled device is located remotely from a user, and wherein the voice-controlled device comprises a gaze tracking component and an indicator component;
determining, using the gaze tracking component of the voice-controlled device, and while the voice control is deactivated, whether a gaze of a user is in a direction of a gaze target, wherein the gaze target comprises a portion of the voice-controlled device in which the indicator component and the gaze tracking component are positioned;
in response to determining that the gaze of the user is not in the direction of the gaze target, and while the voice control is deactiv

Assignees

• Teller Eric
• Aminzade Daniel
• Google Inc

Inventors

• Teller Eric
• Aminzade Daniel

Classifications

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title