Interface elements for managing gesture control

Patent metadata

Field	Value
Publication number	US-9268407-B1
Application number	US-201213649005-A
Country	US
Kind code	B1
Filing date	Oct 10, 2012
Priority date	Oct 10, 2012
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Approaches are described for enabling a user to utilize fingers or other features of both hands to interact with an interface on an electronic device. A user can hold an device such as a mobile phone in a first hand, and can use a finger of the other hand to navigate through an interface or other display, such as to move relative to the device to cause the interface to scroll, zoom, or otherwise adjust. The user can use a thumb or finger of the hand holding the device to select or interact with one or more user-selectable elements, objects, or portions of the interface displayed on a display element of the device, such as by touching the screen over one of the user-selectable elements for at least a determined period of time to activate or change a function associated with a gesture used to interact with the interface.

First claim

What is claimed is:

1. A computer implemented method for determining an input to a mobile computing device, the method comprising:
under control of one or more computer systems configured with executable instructions,
determining, based at least in part on analyzing image information, movement of a fingertip with respect to a display of the mobile computing device;
determining an air gesture based at least in part on the movement of the fingertip becoming closer in distance to the computing device;
determining a selection status of a selectable element of the mobile computing device;
using the selection status to identify a selected navigation action type from a plurality of available navigation action types;
interpreting the air gesture to control a navigation action according to the selected navigation action type; and
determining a zoom level of an image being displayed on the display screen, the zoom level based at least in part on the air gesture.

2. The computer implemented method of claim 1 , wherein the selectable element comprises a selectable graphical user interface element displayed on the display, the method further comprising:
determining, using a sensor on the mobile computing device, at least one position of at least one finger of a hand that is holding the mobile computing device; and
positioning the selectable GUI element on the display screen of the mobile computing device based at least in part on the at least one position.

3. The computer implemented method of claim 1 , wherein the selectable element is a graphical user interface element displayed on a display of the mobile computing device.

4. The computer implemented method of claim 1 , wherein the selected air gesture navigation action type is one of zooming navigation action or scrolling navigation action and wherein the air gesture navigation action is one of zooming or scrolling.

5. A computer implemented method, comprising:
detecting a selection action of a graphical element displayed on a display screen of a computing device, the selection action causing a navigation function to be enabled during a time the selection action is selected;
capturing image information, during the time the selection action is enabled, using a sensor of the computing device;
analyzing the image information to determine a gesture input to the computing device; and
utilizing the gesture input to control the navigation function during the time the selection action is being performed;
detecting a second selection action of a second graphical element displayed on the display screen, the second graphical element being associated with a second function;
enabling the second function;
detecting a second gesture input during a time the second selection action is selected; and
causing, the second function to be performed, wherein the second function is configured to cause at least one of scrolling, zooming, or changing to another displayed portion.

6. The computer implemented method of claim 5 , further comprising:
analyzing the image information to detect at least one position of a fingertip with respect to the computing device;
determining the at least one position of the fingertip with respect to the computing device by analyzing the image information to locate a representation of the fingertip; and
positioning the graphical element on a graphical user interface (GUI) of the computing device based at least in part on the at least one position.

7. The computer implemented method of claim 6 , wherein the fingertip is on a hand of the user that is not holding the computing device.

8. The computer implemented method of claim 5 , further comprising:
detecting a media track being played on the computing device;
determining a wave of a hand of a user of the computing device in one of a left to right direction or a right to left direction relative the computing device, the wave of the hand corresponding to the gesture input;
causing the computing device to skip the media track being played on the computing device based at least in part on the gesture input.

9. The computer implemented method of claim 5 , wherein the graphical element provides a graphical representation of the navigation function that is to be enabled when the selection action is detected.

10. The computer implemented method of claim 5 , further comprising:
detecting the gesture input using camera information from at least one camera of a computing device, the gesture being performed within a detection range of the at least one camera,
detecting a change in orientation of the computing device using at least one of an accelerometer, an electronic compass, an inertial sensor, and an electronic gyroscope;
displaying the graphical element on a display screen of the computing device within a threshold distance to at least one finger of a hand of a user holding the computing device.

11. A computing system, comprising:
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the computing system to:
detect a selection action of a graphical element displayed on a display screen of a computing device, the selection action causing a navigation function to be enabled during a time the selection action is selected;
capture image information, during the time the selection action is enabled, using a sensor of the computing device;
analyze the image information to determine a gesture input to the computing device; and
utilize the gesture input to control the navigation function during the time the selection action is being performed;
detect a second selection action of a second graphical element displayed on the display screen, the second graphical element being associated with a second function;
enable the second function;
detect a second gesture input during a time the second selection action is selected; and
cause, the second function to be performed, wherein the second function is configured to cause the computing device to adjust a displayed portion of a display element by at least one of scrolling the displayed portion, zooming the displayed portion, or changing to display another portion of the displayed portion.

12. The computing system of claim 11 , wherein the instructions when executed further cause the system to:
analyze the image information to detect at least one position of a fingertip with respect to the computing device;
determine the at least one position of the fingertip with respect to the computing device by analyzing the image information to locate a representation of the fingertip; and
position the graphical element on a graphical user interface (GUI) of the computing device based at least in part on the at least one position.

13. The computing system of claim 11 , further comprising: a capacitive sensor configured to sense a change in capacitance near a display screen of the computing device, wherein detecting the selection action includes analyzing a change in capacitance of the capacitive sensor.

14. The computing system of claim 11 , wherein the graphical element provides a graphical representation of the function that is to be enabled when the selection action is detected.

15. The computing system of claim 11 , wherein the instructions, when executed, further cause the system to:
detect the gesture input using camera information from at least one camera of a computing device, the gesture being performed within a detection range of the at least one camera,
detect a change in orientation of the computing device using at least one of an accelerometer, an electronic compass, an inertial

Assignees

• Amazon Tech Inc

Inventors

• Noble Isaac S

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06F3/0488 Primary

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F2203/04806

  Zoom, i.e. interaction techniques or interactors for controlling the zooming operation · CPC title