Point of view determinations for finger tracking

Patent metadata

Field	Value
Publication number	US-8947351-B1
Application number	US-201113246561-A
Country	US
Kind code	B1
Filing date	Sep 27, 2011
Priority date	Sep 27, 2011
Publication date	Feb 3, 2015
Grant date	Feb 3, 2015

Abstract

A user can provide input to a computing device by moving a feature or object, such as a user's finger, within a field of view of at least one imaging element of the computing device. In order to ensure an accuracy of the determined input, the computing device can also attempt to determine a point of view of the user, such as by determining a relative position of the user's face or eyes. By determining a three-dimensional position of a feature and the user's point of view, a three-dimensional vector or other directional information can be determined whereby the intersection of that vector with the computing device indicates an intended location of input corresponding to the feature from the user's point of view.

First claim

What is claimed is:

1. A computer-implemented method of determining input for a computing device, comprising:
capturing a first image using a first camera of the computing device and a second image using a second camera of the computing device, the first and second cameras having overlapping fields of view;
analyzing, using at least one processor of the computing device, the first image to determine a first relative position of a fingertip of a user and a first relative position corresponding to eyes of the user;
analyzing, using the at least one processor of the computing device, the second image to determine a second relative position of the fingertip and a second relative position corresponding to the eyes;
based at least in part upon the first relative position of the fingertip, the second relative position of the fingertip, the first relative position corresponding to the eyes, and the second relative position corresponding to the eyes, determining a three-dimensional fingertip position and a position corresponding to the eyes in three-dimensional space;
calculating, using the at least one processor of the computing device, a vector passing through the three-dimensional fingertip position and the position corresponding to the eyes in three-dimensional space; and
determining, using the at least one processor of the computing device, an input location based at least in part upon the vector and a tilt of the computing device with respect to the user.

2. The computer-implemented method of claim 1 , wherein movement of the fingertip with respect to the computing device is capable of controlling a desired input location with respect to an interface of the computing device from a perspective of the user of the computing device.

3. The computer-implemented method of claim 1 , wherein determining the three-dimensional fingertip position includes:
determining a first feature vector corresponding to the first relative position of the fingertip;
determining a second feature vector corresponding to the second relative position of the fingertip; and
performing vector manipulation to determine the three-dimensional fingertip position based at least in part upon the first feature vector and the second feature vector.

4. The computer-implemented method of claim 1 , wherein determining the position corresponding to the eyes in three-dimensional space includes:
determining a first viewing vector corresponding to the first relative position corresponding to the eyes;
determining a second viewing vector corresponding to the second relative position corresponding to the eyes; and
performing vector manipulation to determine the position corresponding to the eyes in three-dimensional space based at least in part upon the first viewing vector and the second viewing vector.

5. The computer-implemented method of claim 1 , further comprising:
determining a respective dominance of one eye of the user; and
using the respective dominance to further determine the position corresponding to the eyes in three-dimensional space.

6. The computer-implemented method of claim 1 , wherein analyzing the first image to determine the first relative position of the fingertip and the first relative position corresponding to the eyes and analyzing the second image to determine the second relative position of the fingertip and the second relative position corresponding to the eyes includes:
processing the first image and the second image using at least one image recognition or pattern recognition algorithm.

7. The computer-implemented method of claim 1 , further comprising:
detecting the fingertip of the user in the first image and the second image; and
determining an intersection of a respective additional vector for each other detected fingertip for purposes of providing additional input to the computing device.

8. A computer-implemented method of determining input to an electronic device, comprising:
under control of one or more computing devices including executable instructions,
obtaining image information captured using at least two image capture elements of the electronic device;
analyzing the image information to determine a first three-dimensional position of a determined feature of a user;
analyzing the image information to determine a second three-dimensional position corresponding to at least one eye of the user; and
calculating an input location on the electronic device where the user is attempting to provide input to the electronic device, the input location corresponding to a tilt of the electronic device with respect to the user and a vector passing through the first three-dimensional position and the second three-dimensional position.

9. The computer-implemented method of claim 8 , further comprising:
monitoring changes in the input location over time; and
providing a selection input to the electronic device when the input location corresponds to a selectable element of an interface for at least a minimum period of time corresponding to a selection action.

10. The computer-implemented method of claim 8 , further comprising:
monitoring motions of the determined feature;
comparing the motions of the determined feature to a set of input gestures; and
upon one of the motions matching one of the set of input gestures, providing a corresponding input to the electronic device.

11. The computer-implemented method of claim 8 , wherein the determined feature includes a portion of a body of the user or an object being held by the user.

12. The computer-implemented method of claim 8 , wherein the image information includes at least one of ambient light information and reflected infrared radiation information.

13. The computer-implemented method of claim 8 , further comprising:
activating at least one illumination element during obtaining the image information using at least one image capture element.

14. The computer-implemented method of claim 8 , wherein the electronic device includes at least four imaging elements, a first pair of imaging elements capturing the image information in an ambient light detection mode and a second pair capturing the image information in an infrared detection mode.

15. The computer-implemented method of claim 8 , wherein the first three-dimensional position and the second three-dimensional position are determined using at least one of stereoscopic image analysis, feature size analysis, luminance analysis, or distance information from at least one distance determination element.

16. The computer-implemented method of claim 8 , further comprising:
deactivating of obtaining the image information when no determined feature of the user is detected within a specified period of time.

17. The computer-implemented method of claim 8 , wherein the input location corresponds to at least one of magnifying a portion of an interface, adjusting a zoom level of content on the electronic device, selecting an interface element, moving the interface element, or navigating to different portions of the interface.

18. The computer-implemented method of claim 8 , further comprising:
analyzing the image information to determine whether the user is wearing glasses; and
adjusting the second three-dimensional position to account for variations in a determined eye position of the user resulting from the glasses.

19. The computer-implemented method of claim 8 , further comprising:
monitoring adjustments made by the user with respect to the input location; and
adjusting at least one calibration parameter to compensate for the

Assignees

• Noble Isaac S
• Amazon Tech Inc

Inventors

• Noble Isaac S

Classifications

• G06F2203/04108

  Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction · CPC title

• G06F3/042

  by opto-electronic means · CPC title

• G06F3/04883 Primary

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title