Neural network for eye image segmentation and image quality estimation

Patent metadata

Field	Value
Publication number	US-12462394-B2
Application number	US-202318455093-A
Country	US
Kind code	B2
Filing date	Aug 24, 2023
Priority date	Sep 29, 2016
Publication date	Nov 4, 2025
Grant date	Nov 4, 2025

Abstract

Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.

First claim

What is claimed is:

1 . A computer-implemented method for determining eye contours in a semantically segmented eye image, comprising:
receiving a semantically segmented eye image of an eye image comprising a plurality of pixels; determining a pupil contour using the semantically segmented eye image; determining an iris contour using the semantically segmented eye image, wherein,
the pupil contour is determined using a first binary image created based on the semantically segmented eye image, wherein a color value of a first binary image pixel of the first binary image is a fourth color value or a third color value, and/or
the iris contour is determined using a second binary image created based on the semantically segmented eye image, wherein a color value of a second binary image pixel of the second binary image is the third color value or a second color value; and
performing personal biometric identification based the pupil contour and the iris contour.

2 . The computer-implemented method of claim 1 , wherein a first color value is greater than the second color value, wherein the second color value is greater than the third color value, and wherein the third color value is greater than the fourth color value.

3 . The computer-implemented method of claim 1 , wherein determining the pupil contour using the semantically segmented eye image, comprises:
determining a pupil contour border; removing a plurality of pixels from the pupil contour border; and determining the pupil contour as an ellipse from remaining pixels of the pupil contour border.

4 . The computer-implemented method of claim 3 , wherein determining a pupil contour border, comprises:
determining contours in the first binary image; and selecting a longest contour of the determined contours in the first binary image as a pupil contour border.

5 . The computer-implemented method of claim 4 , wherein the color value of the first binary image pixel of the first binary image is the fourth color value if a corresponding pixel in the semantically segmented eye image has a value greater than or equal to the fourth color value, and the third color value if the corresponding pixel in the semantically segmented eye image has a value not greater than or equal to the fourth color value.

6 . The computer-implemented method of claim 4 , comprising:
determining a pupil contour points bounding box enclosing the pupil contour border; computing a pupil points area size as a diagonal of the pupil contours points bounding box; and determining a pupil contour threshold based on the pupil points area size.

7 . The computer-implemented method of claim 6 , wherein the pupil contour threshold is a fraction multiplied by the pupil points area size, and wherein the fraction is in a range from 0.02 to 0.20.

8 . The computer-implemented method of claim 3 , comprising creating a third binary image comprising a plurality of pixels, wherein a color value of a third binary image pixel of the plurality of pixels of the third binary image is the third color value or the second color value.

9 . The computer-implemented method of claim 8 , wherein the color value of the third binary image pixel of the plurality of pixels of the third binary image is the third color value if a corresponding pixel in the semantically segmented eye image has a value greater than or equal to the third color value, and the second color value if the corresponding pixel in the semantically segmented eye image has a value not greater than or equal to the third color value.

10 . The computer-implemented method of claim 8 , wherein removing a plurality of pixels from the pupil contour border comprises, for a pupil contour border pixel of the pupil contour border:
determining a closest pixel in the third binary image that has a color value of the second color value and that is closest to the pupil contour border pixel; determining a distance between the pupil contour border pixel and the closest pixel in the third binary image; and removing the pupil contour border pixel from the pupil contour border if the distance between the pupil contour border pixel and the closest pixel in the third binary image is smaller than a pupil contour threshold.

11 . The computer-implemented method of claim 1 , wherein determining the iris contour using the semantically segmented eye image, comprises:
determining an iris contour border; removing a plurality of pixels from the iris contour border; and determining the iris contour as an ellipse from remaining pixels of the iris contour border.

12 . The computer-implemented method of claim 11 , wherein determining the iris contour border, comprises:
determining contours in the second binary image; and selecting a longest contour of the determined contours in the second binary image as an iris contour border.

13 . The computer-implemented method of claim 12 , comprising:
determining an iris contour points bounding box enclosing the iris contour border; computing an iris points area size as a diagonal of the iris contours points bounding box; and determining an iris contour threshold based on the iris points area size.

14 . The computer-implemented method of claim 13 , wherein the iris contour threshold is a fraction multiple by the iris points area size, and wherein the fraction is in a range from 0.02 to 0.20.

15 . The computer-implemented method of claim 11 , wherein a color value of the second binary image pixel of the plurality of pixels of the second binary image is the third color value if a corresponding pixel in the semantically segmented eye image has a value greater than or equal to the third color value, and the second color value if the corresponding pixel in the semantically segmented eye image has a value not greater than or equal to the third color value.

16 . The computer-implemented method of claim 11 comprising creating a fourth binary image comprising a plurality of pixels, wherein a color value of a fourth binary image pixel of the plurality of pixels of the fourth binary image is the second color value or a first color value.

17 . The computer-implemented method of claim 16 , wherein a color value of a fourth binary image pixel of the plurality of pixels of the fourth binary image is the second color value if a corresponding pixel in the semantically segmented eye image has a value greater than or equal to the second color value, and the first color value if the corresponding pixel in the semantically segmented eye image has a value not greater than or equal to the second color value.

18 . The computer-implemented method of claim 16 , wherein removing a plurality of pixels from the iris contour border comprises, for an iris contour border pixel of the iris contour border:
determining a closest pixel in the fourth binary image that has a color value of the first color value and that is closest to the iris contour border pixel; determining a distance between the iris contour border pixel and the closest pixel in the fourth binary image; and removing the iris contour border pixel from the iris contour border if the distance between the iris contour border pixel and the closest pixel in the fourth binary image is smaller than an iris contour threshold.

19 . The computer-implemented method of claim 1 comprising determining a binary mask to cover an irrelevant area in the semantically segmented eye image.

20 . The computer-implemented method of claim 19 wherein determining the binary mask to cover

Assignees

• Magic Leap Inc

Inventors

• Spizhevoy Alexey
• Kaehler Adrian
• Badrinarayanan Vijay

Classifications

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06T7/10 Primary

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title

• G06V10/82

  using neural networks · CPC title