Image segmentation of a video stream

Patent metadata

Field	Value
Publication number	US-9830708-B1
Application number	US-201514884536-A
Country	US
Kind code	B1
Filing date	Oct 15, 2015
Priority date	Oct 15, 2015
Publication date	Nov 28, 2017
Grant date	Nov 28, 2017

Abstract

Systems, devices, media, and methods are presented for segmenting an image of a video stream with a client device, binarizing an area of interest within one or more image, identifying an initial pupil location and an initial iris radius, and determining a final pupil location and a final iris radius. Some embodiments enable the client device to perform one or more operations within a user interface based on the image segmentation.

First claim

What is claimed is:

1. A computer implemented method for segmenting a video stream, comprising:
determining an approximate location of an eye within a video stream comprising a set of images to identify an area of interest within one or more images of the set of images;
binarizing the area of interest within the one or more images to generate a binarized image from the video stream;
identifying an initial pupil location of a pupil of the eye within the binarized image by identifying a set of first pixels within the binarized image, the set of first pixels having a first value and having an area greater than a predetermined percentage of the area of interest and determine an approximate center of mass of the set of first pixels;
determining an initial iris radius for the eye; and
based on the initial pupil location and the initial iris radius, determining a final pupil location and a final iris radius.

2. The method of claim 1 further comprising:
cropping one or more images of the video stream to isolate the area of interest within the one or more images of the video stream.

3. The method of claim 1 , wherein the area of interest is binarized using a red channel of the one or more images of the set of images.

4. The method of claim 1 further comprising:
determining a set of first pixels and a set of second pixels within the binarized image, the set of first pixels representing the iris and pupil and having a first value, the set of second pixels representing a noise region having a second value and positioned within the set of first pixels, an area of the noise region being smaller than a predetermined percentage of the area of interest; and
converting the second value of the set of second pixels from the noise region to the first value.

5. The method of claim 1 , wherein determining the final pupil location further comprises:
identifying a perimeter for the set of first pixels, the perimeter including a set of perimeter pixels having a first value;
smoothing the perimeter, by modifying a value of one or more perimeter pixels of the set of perimeter pixels based on values of a set of proximate pixels, to generate a smoothed set of first pixels; and
determining the final pupil location based on the smoothed set of first pixels.

6. The method of claim 1 , wherein determining the initial iris radius further comprises:
identifying a first distance from a first corner of the eye to an outer edge of the iris and a second distance from a second corner of the eye to the outer edge of the iris; and
determining the initial iris radius based on the first distance and the second distance.

7. The method of claim 6 , wherein determining the final iris radius further comprises:
identifying a perimeter for a set of first pixels within the binarized image, the set of first pixels having a first value and representing an area of the iris and the pupil;
smoothing the perimeter by modifying a value of one or more perimeter pixels of the set of perimeter pixels based on values of a set of proximate pixels, to generate a smoothed set of first pixels; and
determining the final iris radius based on the smoothed set of first pixels.

8. The method of claim 1 further comprising:
forming a binary mask for the eye, the binary mask identifying a set of pixels associated with the iris of the eye.

9. The method of claim 1 , wherein the video stream is a first portion of a video stream, and further comprising:
tracking at least one of the final pupil location and the final iris radius across one or more images included in a second portion of the video stream.

10. The method of claim 1 further comprising:
detecting an obstruction of the eye within the area of interest in one or more image within the video stream to identify one or more obstructed images and one or more unobstructed images in the video stream; and
binarizing the area of interest within the one or more unobstructed images to generate a binarized image from the video stream.

11. A system, comprising:
one or more processors; and
a non-transitory machine-readable storage medium storing processor executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
determining an approximate location of an eye within a video stream comprising a set of images to identify an area of interest within one or more images of the set of images;
binarizing the area of interest within the one or more images to generate a binarized image from the video stream;
identifying an initial pupil location of a pupil of the eye within the binarized image by identifying a set of first pixels within the binarized image, the set of first pixels having a first value and having an area greater than a predetermined percentage of the area of interest and determine an approximate center of mass of the set of first pixels;

determining an initial iris radius for the eye; and
based on the initial pupil location and the initial iris radius, determining a final pupil location and a final iris radius.

12. The system of claim 11 , wherein the processor executable instructions cause the one or more processors to perform operations comprising:
determining a set of first pixels and a set of second pixels within the binarized image, the set of first pixels representing the iris and pupil and having a first value, the set of second pixels representing a noise region having a second value and positioned within the set of first pixels, an area of the noise region being smaller than a predetermined percentage of the area of interest; and
converting the second value of the set of second pixels from the noise region to the first value.

13. The system of claim 11 , wherein in determining the final u location, the processor executable instructions cause the one or more processors to perform operations comprising:
identifying a perimeter for the set of first pixels, the perimeter including a set of perimeter pixels having a first value;
smoothing the perimeter, by modifying a value of one or more perimeter pixels of the set of perimeter pixels based on values of a set of proximate pixels, to generate a smoothed set of first pixels; and
determining the final pupil location based on the smoothed set of first pixels.

14. The system of claim 11 , wherein in determining the initial iris radius, the processor executable instructions cause the one or more processors to perform operations comprising:
identifying a first distance from a first corner of the eye to an outer edge of the iris and a second distance from a second corner of the eye to the outer edge of the iris; and
determining the initial iris radius based on the first distance and the second distance.

15. The system of claim 14 , wherein in determining the final iris radius, the processor executable instructions cause the one or more processors to perform operations comprising:
identifying a perimeter for a set of first pixels within the binarized image, the set of first pixels having a first value and representing an area of the iris and the pupil;
smoothing the perimeter, by modifying a value of one or more perimeter pixels of the set of perimeter pixels based on values of a set of proximate pixels, to generate a smoothed set of first pixels; and
determining the final iris radius based on the smoothed set of first pixels.

16. The system of claim 11 , wherein the processor executable instructions cause the one or more processors to perform operations comprising:
tracking at least one of the final pupil location and the final iris

Assignees

• Snap Inc

Inventors

• Poliakov Fedir

Classifications

• G06T7/62

  of area, perimeter, diameter or volume · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06T7/136

  involving thresholding · CPC title

• G06T7/66

  of image moments or centre of gravity · CPC title

• G06T2207/30041

  Eye; Retina; Ophthalmic · CPC title