Efficient forest sensing based eye tracking

Patent metadata

Field	Value
Publication number	US-9514364-B2
Application number	US-201414290408-A
Country	US
Kind code	B2
Filing date	May 29, 2014
Priority date	May 29, 2014
Publication date	Dec 6, 2016
Grant date	Dec 6, 2016

Abstract

Methods, systems, computer-readable media, and apparatuses for novel eye tracking methodologies are presented. Specifically, after an initial determination of a person's eyes within a field of view (FOV), methods of the present disclosures may track the person's eyes even with part of the face occluded, and may quickly re-acquire the eyes even if the person's eyes exit the FOV. Each eye may be tracked individually, at a faster rate of eye tracking due to the novel methodology, and successful eye tracking even at low image resolution and/or quality is possible. In some embodiments, the eye tracking methodology of the present disclosures includes a series of sub-tracker techniques, each performing different eye-tracking functions that, when combined, generate a highest-confidence location of where the eye has moved to in the next image frame.

First claim

What is claimed is:

1. A method for tracking an eye, the method comprising:
receiving a first image comprising a face and the eye;
determining an initial position of the eye in a field of view (FOV) based on the received first image;
receiving a second image comprising a portion of the face including the eye; and
tracking a change in a position of the eye from the initial position of the eye based on the received second image, wherein tracking the change is based at least on a convergence of results outputted by a plurality of tracking modules, the plurality of tracking modules including (1) a first tracking module configured to track an iris portion of the eye, (2) a second tracking module configured to generate a portion of the face that includes the eye and a surrounding region having features other than the eye, and (3) a third tracking module configured to compare differences between the first received image and the second received image;
wherein the convergence of results is based on at least one high confidence position of the eye generated by each one of the plurality of tracking modules.

2. The method of claim 1 , further comprising generating a plurality of positive image samples, each representing a high-confidence detection of the eye.

3. The method of claim 2 , further comprising generating a plurality of negative image samples, each representing a high-confidence detection of a portion of the first image other than the eye.

4. The method of claim 3 , wherein tracking a change in the position of the eye is based further on the plurality of positive image samples and the plurality of negative image samples.

5. The method of claim 1 , further comprising generating an intersection bounding box comprising the eye and a subset of the portion of the face, and wherein tracking a change in the position of the eye is based further on the intersection bounding box.

6. The method of claim 1 , further comprising detecting motion of the eye between the first image and the second image.

7. The method of claim 6 , wherein tracking a change in the position of the eye is based further on the detected motion of the eye.

8. The method of claim 1 , wherein the tracking step is performed in real-time.

9. An apparatus for tracking an eye, the apparatus comprising:
a camera configured to capture one or more images;
a memory comprising a plurality of tracking modules; and
a processor coupled to the camera and the memory, the processor operable, when the one or more tracking modules are executed, to:
receive, via the camera, a first image comprising a face and the eye;
determine an initial position of the eye in a field of view (FOV) based on the received first image;
receive, via the camera, a second image comprising a portion of the face including the eye; and
track a change in a position of the eye from the initial position of the eye based on the received second image, wherein tracking the change is based at least on a convergence of results outputted by the plurality of tracking modules, the plurality of tracking modules including (1) a first tracking module configured to track an iris portion of the eye, (2) a second tracking module surrounding region having features other than the eye, and (3) a third tracking module configured to compare differences between the first received image and the second received image;

wherein the convergence of results is based on at least one high confidence position of the eye generated by each one of the plurality of tracking modules.

10. The apparatus of claim 9 , wherein the processor is further operable, when the one or more tracking modules are executed, to generate a plurality of positive image samples, each representing a high-confidence detection of the eye.

11. The apparatus of claim 10 , wherein the processor is further operable, when the one or more tracking modules are executed, to generate a plurality of negative image samples, each representing a high-confidence detection of a portion of the first image other than the eye.

12. The apparatus of claim 11 , wherein tracking a change in the position of the eye is based further on the plurality of positive image samples and the plurality of negative image samples.

13. The apparatus of claim 9 , wherein the processor is further operable, when the one or more tracking modules are executed, to generate an intersection bounding box comprising the eye and a subset of the portion of the face, and wherein tracking a change in the position of the eye is based further on the intersection bounding box.

14. The apparatus of claim 9 , wherein the processor is further operable, when the one or more tracking modules are executed, to detection motion of the eye between the first image and the second image.

15. The apparatus of claim 14 , wherein tracking a change in the position of the eye is based further on the detected motion of the eye.

16. The apparatus of claim 9 , wherein the tracking is performed in real-time.

17. An apparatus for tracking an eye, the apparatus comprising:
means for receiving a first image comprising a face and the eye;
means for determining an initial position of the eye in a field of view (FOV) based on the received first image;
means for receiving a second image comprising a portion of the face including the eye; and
means for tracking a change in a position of the eye from the initial position of the eye based on the received second image, wherein tracking the change is based at least on a means for converging results outputted by a plurality of tracking modules, the plurality of tracking modules including (1) a first tracking module configured to track an iris portion of the eye, (2) a second tracking module configured to generate a portion of the face that includes the eye and a surrounding region having features other than the the eye, and (3) a third tracking module configured to compare differences between the first received image and the second received image;
wherein the means for converging results is based on at least one high confidence position of the eye generated by each one of the plurality of tracking modules.

18. The apparatus of claim 17 , further comprising means for generating a plurality of positive image samples, each representing a high-confidence detection of the eye.

19. The apparatus of claim 18 , further comprising means for generating a plurality of negative image samples, each representing a high-confidence detection of a portion of the first image other than the eye.

20. The apparatus of claim 19 , wherein tracking a change in the position of the eye is based further on the plurality of positive image samples and the plurality of negative image samples.

21. The apparatus of claim 17 , further comprising means for generating an intersection bounding box comprising the eye and a subset of the portion of the face, and wherein tracking a change in the position of the eye is based further on the intersection bounding box.

22. The apparatus of claim 17 , further comprising means for detecting motion of the eye between the first image and the second image.

23. The apparatus of claim 22 , wherein tracking a change in the position of the eye is based further on the detected motion of the eye.

24. A processor-readable non-transitory computer-readable medium comprising processor readable instructions configured to cause a processor to:
receive a first image comprising a face and an eye;
determine an initia

Assignees

• Qualcomm Inc

Inventors

• Sheng Tao
• Dos Remedios Alwyn

Classifications

• G06V40/193 Primary

  Preprocessing; Feature extraction · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06V10/443

  by matching or filtering · CPC title

• G06K9/46

  Physics · mapped topic

• G06K9/0061 Primary

  Physics · mapped topic