Eye detecting method, eye detecting device, and non-transitory computer-readable recording medium recording eye detecting program

Patent metadata

Field	Value
Publication number	US-11961328-B2
Application number	US-202217828440-A
Country	US
Kind code	B2
Filing date	May 31, 2022
Priority date	Dec 22, 2020
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

An eye detecting device is configured to: acquire a color image including a face of a person taken by an image taking device; generate a grayscale image by multiplying each of a red component value, a green component value, and a blue component value of each pixel of the color image by a predetermined ratio according to characteristics of a lens of glasses that the person wears; detect an eye of the person from the grayscale image; and output eye information on the eye of the person.

First claim

The invention claimed is:

1. An eye detecting method comprising:
acquiring, by a processor, a color image including a face of a person taken by an image taking device;
generating, by the processor, a grayscale image by multiplying each of a red component value, a green component value, and a blue component value of each pixel of the color image by a predetermined ratio according to characteristics of a lens of glasses that the person wears;
detecting, by the processor, an eye of the person from the grayscale image;
generating, by the processor, a binary image from the color image;
extracting, by the processor, from the binary image an estimated glasses region where the glasses are estimated to be present;
determining, by the processor, whether the person is wearing the glasses, based on a length in a horizontal direction of a white region where a plurality of white pixels continue in the estimated glasses region;
detecting, by the processor, the eye of the person from the color image upon determining that the person is not wearing the glasses; and
outputting, by the processor, eye information on the eye of the person,
wherein the grayscale image is generated by the processor upon determining that the person is wearing the glasses.

2. The eye detecting method according to claim 1 , wherein
the predetermined ratio by which the red component value is multiplied is higher than the predetermined ratio by which the green component value is multiplied and the predetermined ratio by which the blue component value is multiplied.

3. The eye detecting method according to claim 2 , wherein
the predetermined ratio by which the blue component value is multiplied is zero.

4. The eye detecting method according to claim 1 , further comprising:
calculating, by the processor, an average luminance value of an eye region including the eye of the person;
determining, by the processor, whether the average luminance value is higher than a threshold; and
outputting, by the processor, reflection information indicating that the lens of the glasses reflects external light or a background, upon determining that the average luminance value is higher than the threshold.

5. The eye detecting method according to claim 1 , further comprising:
detecting, by the processor, a face region including the face of the person from the color image;
calculating, by the processor, an average luminance value of the face region;
determining, by the processor, that the eye of the person is detectable, when the average luminance value is higher than a threshold; and
determining, by the processor, that the eye of the person is undetectable, when the average luminance value is not higher than the threshold.

6. The eye detecting method according to claim 5 , further comprising:
notifying the person, by the processor, that the eye of the person is undetectable, upon determining that the eye of the person is undetectable.

7. The eye detecting method according to claim 1 , further comprising:
generating, by the processor, a first image in which a pixel having a gradation value smaller than a threshold is represented at a first luminance value and a pixel having a gradation value not smaller than the threshold is represented at a second luminance value, after the grayscale image is binarized;
generating, by the processor, a second image by replacing a pixel which has the second luminance value in a first luminance region having the first luminance value in the first image and satisfies a predetermined condition with a pixel having the first luminance value;
calculating, by the processor, iris information including information indicative of at least one of a position of an iris of the person or a size of the iris of the person using the second image; and
outputting, by the processor, the iris information.

8. An eye detecting device comprising:
a processor; and
a non-transitory computer-readable medium having stored thereon executable instructions that, when executed by the processor, cause the eye detecting device to function as:
an acquiring part for acquiring a color image including a face of a person taken by an image taking device;
a generating part for generating a grayscale image by multiplying each of a red component value, a green component value, and a blue component value of each pixel of the color image by a predetermined ratio according to characteristics of a lens of glasses that the person wears;
a detecting part for detecting an eye of the person from the grayscale image;
wherein the generating part is also for generating a binary image from the color image;
a determining part for extracting from the binary image an estimated glasses region where the glasses are estimated to be present;
wherein the determining part is also for determining whether the person is wearing the glasses, based on a length in a horizontal direction of a white region where a plurality of white pixels continue in the estimated glasses region;
wherein the detecting part is also for detecting the eye of the person from the color image upon determining that the person is not wearing the glasses; and
an outputting part for outputting eye information on the eye of the person,
wherein the eye detecting device is configured to generate the grayscale image upon determining that the person is wearing the glasses.

9. A non-transitory computer-readable medium having stored thereon executable instructions that, when executed by a processor of an eye detecting device, cause the eye detecting device to:
acquire a color image including a face of a person taken by an image taking device;
generate a grayscale image by multiplying each of a red component value, a green component value, and a blue component value of each pixel of the color image by a predetermined ratio according to characteristics of a lens of glasses that the person wears;
detect an eye of the person from the grayscale image;
generate a binary image from the color image;
extract from the binary image an estimated glasses region where the glasses are estimated to be present;
determine whether the person is wearing the glasses, based on a length in a horizontal direction of a white region where a plurality of white pixels continue in the estimated glasses region;
detect the eye of the person from the color image upon determining that the person is not wearing the glasses; and
output eye information on the eye of the person,
wherein the grayscale image is generated by the eye detecting device upon determining that the person is wearing the glasses.

Assignees

• Swallow Incubate Co Ltd
• Panasonic Holdings Corp

Inventors

• Ohno Toshikazu

Classifications

• G06V40/193 Primary

  Preprocessing; Feature extraction · CPC title

• G06V10/46

  Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56) · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06T7/00

  Image analysis · CPC title

• G06T7/11

  Region-based segmentation · CPC title