Eye gaze detection method, eye gaze detection device, and non-transitory computer readable storage medium

Patent metadata

Field	Value
Publication number	US-12327372-B2
Application number	US-202217741632-A
Country	US
Kind code	B2
Filing date	May 11, 2022
Priority date	Nov 13, 2019
Publication date	Jun 10, 2025
Grant date	Jun 10, 2025

Abstract

An eye gaze detection method includes: acquiring image data of a face of a person; detecting, from the image data, a left eye region including a left eye of the person and a right eye region including a right eye of the person; detecting, from the image data of the face, at least one of an ear region including an ear of the person and a hair region including hair of the person, when only one eye region of the right eye region or the left eye region is detected; detecting information indicating an orientation of the face based on information indicating a position of one eye included in the one eye region and information indicating the at least one; and detecting information indicating an eye gaze of the person based on the information indicating the position of the one eye and the information indicating the orientation of the face.

First claim

The invention claimed is:

1. An eye gaze detection method in an eye gaze detection device, the eye gaze detection method comprising:
acquiring image data of a face of a person;
detecting, from the image data of the face, a left eye region including a left eye of the person and a right eye region including a right eye of the person;
detecting, from the image data of the face, a certain site of the face existing on an outer corner of one eye relative to a position of an inner corner of the one eye, wherein the certain site includes an ear region including an ear of the person or a hair region including hair of the person, when only one eye region of the right eye region or the left eye region is detected;
detecting a contour point indicating a point on a contour of the face existing on the inner corner of the one eye relative to a position of the outer corner of the one eye based on the image data of the face and information indicating a position of the one eye;
detecting information indicating an orientation of the face based on a first distance from the position of the one eye to the contour point and a second distance from the position of the one eye to a position of the certain site; and
detecting information indicating an eye gaze of the person based on the information indicating the position of the one eye and the information indicating the orientation of the face.

2. The eye gaze detection method according to claim 1 , comprising:
detecting the ear region from the image data of the face when only the one eye region is detected; and
detecting information indicating an orientation of the face based on the first distance from the position of the one eye to the contour point and the second distance from the position of the one eye to the position of the certain site.

3. The eye gaze detection method according to claim 2 , comprising:
detecting the hair region from the image data of the face when having failed to detect the ear region, and
detecting information indicating an orientation of the face based on the information indicating the position of the one eye and the information indicating the position of the hair.

4. The eye gaze detection method according to claim 1 , wherein
the certain site includes the hair region including hair of the person, the eye gaze detection method comprising:
detecting information indicating an orientation of the face based on the first distance from the position of the one eye to the contour point and the second distance from the position of the one eye to a point in the hair region.

5. The eye gaze detection method according to claim 1 , comprising:
detecting information indicating an orientation of the face from the image data of the face when the right eye region and the left eye region are detected; and
detecting information indicating an eye gaze of the person based on information indicating a position of a right eye included in the right eye region and a position of a left eye included in the left eye region and the information indicating the orientation of the face.

6. An eye gaze detection device comprising a processor configured to perform:
acquiring image data of a face of a person;
detecting, from the image data of the face, a left eye region including a left eye of the person and a right eye region including a right eye of the person;
detecting, from the image data of the face, a certain site of the face existing on an outer corner of one eye relative to a position of an inner corner of the one eye, wherein the certain site includes an ear region including an ear of the person or a hair region including hair of the person, when only one eye region of the right eye region or the left eye region is detected;
detecting a contour point indicating a point on a contour of the face existing on the inner corner of the one eye relative to a position of the outer corner of the one eye based on the image data of the face and information indicating a position of the one eye and detecting information indicating an orientation of the face based on a first distance from the position of the one eye to the contour point and a second distance from the position of the one eye to a position of the certain site; and
detecting information indicating an eye gaze of the person based on the information indicating the position of the one eye and the information indicating the orientation of the face.

7. A non-transitory computer readable storage medium storing a control program for an eye gaze detection device, the control program causing a computer included in the eye gaze detection device to execute:
acquiring image data of a face of a person;
detecting, from the image data of the face, a left eye region including a left eye of the person and a right eye region including a right eye of the person;
detecting, from the image data of the face, a certain site of the face existing on an outer corner of one eye relative to a position of an inner corner of the one eye, wherein the certain site includes an ear region including an ear of the person or a hair region including hair of the person, when only one eye region of the right eye region or the left eye region is detected;
detecting a contour point indicating a point on a contour of the face existing on the inner corner of the one eye relative to a position of the outer corner of the one eye based on the image data of the face and information indicating a position of the one eye and detecting information indicating an orientation of the face based on a first distance from the position of the one eye to the contour point and a second distance from the position of the one eye to a position of the certain site; and
detecting information indicating an eye gaze of the person based on the information indicating the position of the one eye and the information indicating the orientation of the face.

Assignees

• Swallow Incubate Co Ltd
• Panasonic Holdings Corp

Inventors

• Ohno Toshikazu

Classifications

• G06T2207/30201

  Face · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/165

  using facial parts and geometric relationships · CPC title

• G06T1/00

  General purpose image data processing · CPC title