Pupil detection

Patent metadata

Field	Value
Publication number	US-9355315-B2
Application number	US-201414340279-A
Country	US
Kind code	B2
Filing date	Jul 24, 2014
Priority date	Jul 24, 2014
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

Embodiments that relate to determining an estimated pupil region of an eye are disclosed. In one embodiment a method includes receiving an image of an eye, with the image comprising a plurality of pixels. A rough pupil region is generated using at least a subset of the plurality of pixels. A plurality of pupil boundary point candidates are extracted from the rough pupil region, with each of the candidates weighted based on color values of at least two neighbor pixels. A parametric curve may be fitted to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

First claim

The invention claimed is:

1. A method for determining an estimated pupil region of an eye of a user, the method comprising:
receiving an image of the eye of the user, the image comprising a plurality of pixels;
generating a rough pupil region using at least a subset of the plurality of pixels;
extracting from the rough pupil region a plurality of pupil boundary point candidate pixels;
for each of the pupil boundary point candidate pixels, weighting the pupil boundary point candidate pixel based on color values of at least two neighbor pixels of the pupil boundary point candidate pixel to generate weighted pupil boundary point candidates; and
fitting a parametric curve to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

2. The method of claim 1 , wherein weighting each of the pupil boundary point candidate pixels based on color values of at least two neighbor pixels further comprises:
determining a color value of each of a plurality of inner pixels located between the pupil boundary point candidate pixel and an estimated center of the rough pupil region along a ray extending from the estimated center through the pupil boundary point candidate pixel; and
determining a color value of each of a plurality of outer pixels located along the ray, the plurality of outer pixels located on an opposite side of the pupil boundary point candidate pixel from the inner pixels.

3. The method of claim 2 , wherein weighting each of the pupil boundary point candidate pixels based on color values of at least two neighbor pixels further comprises averaging the color values of the plurality of inner pixels, and averaging the color values of the plurality of outer pixels.

4. The method of claim 2 , wherein weighting each of the pupil boundary point candidate pixels further comprises:
clustering the pupil boundary point candidate pixels in a two-dimensional color space according to an average color value of their inner pixels and an average color value of their outer pixels; and
assigning a weight to each of the pupil boundary point candidate pixels based on a distance of each of the pupil boundary point candidate pixels from a center of a dominant cluster in the two-dimensional color space.

5. The method of claim 1 , wherein fitting the parametric curve to the weighted pupil boundary point candidates further comprises:
minimizing a cost function that is computed using each of the weighted pupil boundary point candidates.

6. The method of claim 5 , wherein one or more elements contributing to the cost function comprise one or more of a discrepancy between an image-space gradient and a parametric curve normal for each of the weighted pupil boundary point candidates, and a distance of each of the weighted pupil boundary point candidates from the parametric curve.

7. The method of claim 1 , wherein generating a rough pupil region further comprises extracting one or more stable regions from the image.

8. The method of claim 7 , wherein extracting one or more stable regions from the image further comprises identifying one or more maximally stable extremal regions.

9. The method of claim 1 , wherein generating a rough pupil region further comprises using one or more classifiers to classify each of the pixels of the subset of the plurality of pixels into one of a plurality of classification regions.

10. The method of claim 1 , wherein the rough pupil region comprises one or more connected components, and the method further comprises scoring each of the connected components using a plurality of heuristics corresponding to different pupil characteristics.

11. A system for determining an estimated pupil region of an eye of a user, the system comprising:
a computing device; and
a pupil detection module executed by a processor of the computing device, the pupil detection module configured to:
receive an image of the eye of the user, the image comprising a plurality of pixels;
generate a rough pupil region using at least a subset of the plurality of pixels;
extract from the rough pupil region a plurality of pupil boundary point candidate pixels;
for each of the pupil boundary point candidate pixels, weight the pupil boundary point candidate pixel based on color values of at least two neighbor pixels of the pupil boundary point candidate pixel to generate weighted pupil boundary point candidates; and
fit a parametric curve to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

12. The system of claim 11 , wherein to weight each of the pupil boundary point candidate pixels based on the color values of at least two neighbor pixels, the pupil detection module is further configured to:
determine a color value of each of a plurality of inner pixels located between the pupil boundary point candidate pixel and an estimated center of the rough pupil region along a ray extending from the estimated center through the pupil boundary point candidate pixel; and
determine a color value of each of a plurality of outer pixels located along the ray, the plurality of outer pixels located on an opposite side of the pupil boundary point candidate pixel from the inner pixels.

13. The system of claim 12 , wherein to weight each of the pupil boundary point candidate pixels based on the color values of at least two neighbor pixels, the pupil detection module is further configured to:
average the color values of the plurality of inner pixels, and
average the color values of the plurality of outer pixels.

14. The system of claim 12 , wherein to weight each of the pupil boundary point candidate pixels based on the color values of at least two neighbor pixels, the pupil detection module is further configured to:
cluster the pupil boundary point candidate pixels in a two-dimensional color space according to their inner pixel values and their outer pixels values; and
assign a weight to each of the pupil boundary point candidate pixels based on a distance of each of the pupil boundary point candidate pixels from a center of a dominant cluster in the two-dimensional color space.

15. The system of claim 11 , wherein to fit the parametric curve to the weighted pupil boundary point candidates, the pupil detection module is further configured to minimize a cost function that is computed using each of the weighted pupil boundary point candidates.

16. The system of claim 15 , wherein one or more elements contributing to the cost function comprise one or more of a discrepancy between an image-space gradient and a parametric curve normal for each of the weighted pupil boundary point candidates, and a distance of each of the weighted pupil boundary point candidates from the parametric curve.

17. The system of claim 11 , wherein to generate a rough pupil region the pupil detection module is further configured to extract one or more stable regions from the image.

18. The system of claim 17 , wherein to extract one or more stable regions from the image the pupil detection module is further configured to identify one or more maximally stable extremal regions.

19. The system of claim 11 , wherein to generate a rough pupil region the pupil detection module is further configured to use one or more classifiers to classify each of the pixels of the subset of the plurality of pixels into one of a plurality of classification regions.

20. A method for determining an estimated pupil region of an eye of a user, the method comprising:
receiving an image of the eye of t

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Vugdelija Milan
• Nijemcevic Djordje
• Nister David Zachris
• Mei Christopher Maurice
• Gatla Chandra Sekhar

Classifications

• G06T7/73

  using feature-based methods · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/30201

  Face · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06V40/193 Primary

  Preprocessing; Feature extraction · CPC title