Iris and pupil-based gaze estimation method for head-mounted device

Patent metadata

Field	Value
Publication number	US-10353465-B2
Application number	US-201615770354-A
Country	US
Kind code	B2
Filing date	Dec 23, 2016
Priority date	Jun 8, 2016
Publication date	Jul 16, 2019
Grant date	Jul 16, 2019

Abstract

The present invention discloses a gaze estimation method for a headset device based, on iris and pupil. In the method, only a single camera, a single infrared light source and a central calibration point which is each provided within four average regions at a screen are used, 3D modeling of the pupil and iris is performed through an extracted 2D central feature of eye movement, and a 3D gaze direction vector is established and combined with angle information of the gaze direction, so as to estimate a gaze point position of human eyes. Fully taking a characteristic of applying the headset device into account, the present invention guarantees an accuracy of the gaze point estimation in the headset device and meanwhile greatly reduces the complexity of an entire system structure and a calibration process, and provides a solution with high accuracy and little complexity for the gaze tracking in the headset device.

First claim

What is claimed:

1. A gaze estimation method for a headset device based on iris and pupil, which only needs a single camera, a single infrared light source and a central calibration point which is each provided within four average regions at a screen, characterized in that the method includes following steps:
(1) eye movement feature extraction: inside the headset device, human eyes are close to the screen and the camera, a concept of first acquiring an outline of the eye movement feature and then positioning an eye movement feature center is used for a human eye image captured by the camera under a near-infrared light, and as for segmented feature regions of the pupil and iris, 2D central parameters of the pupil and iris are acquired by using an edge detection and an ellipse fitting algorithm;
(2) 3D modeling of the eye movement feature: relative positions and distances between the human eyes and the screen as well as the human eyes and the camera remain constant, and meanwhile the pupil and the iris are not located in the same plane and the iris is in front of the pupil; according to a conversion formula from an image pixel coordinate system to a camera coordinate system, 2D image information of the human eyes, which is captured by the camera when the human eyes observe the central calibration points within four regions at the screen, is combined with position information of the calibration points, and a 3D gaze mapping model is obtained by 3D modeling an eye movement feature vector, i.e. 3D central coordinate parameters of the pupil and iris when the human eyes gaze at the central calibration points within four regions are obtained; and
(3) estimation of a gaze point position: when the human eyes gaze at any point on the screen, the central parameters of the pupil and iris extracted from the 2D human eye image are compared with a 2D center of four calibration points, to position a gaze region of the human eyes, by using the established 3D gaze mapping model to calculate 3D coordinates of a pupil center and an iris center within the gaze region, a pupil-iris 3D gaze direction vector is obtained and combined with angle information of the gaze estimation to obtain a final gaze point position of the human eyes.

2. The gaze estimation method for the headset device based on the iris and the pupil according to claim 1 , wherein the step (1) includes:
a. as for an input human eye image, a template matching algorithm is used to initially position a right eye region;
b. as for a feature extraction part of the pupil center, first an OTSU algorithm which is combined with a compensation algorithm using a histogram peak as a search threshold, is used to achieve an automatic segmentation of the pupil region, and a maximum connected region search algorithm is used to eliminate a noise block on a binary image of the pupil, an edge of the pupil is obtained by using a Sobel edge detection, and finally central parameter information of the pupil is acquired through a RANSAC ellipse fitting algorithm;
c. as for a feature extraction part of the iris center, an iris-interested region is determined by using a feature extraction center of the iris as a center of a ROI region, and a binary image of the iris is acquired by using a digital morphology combined with a histogram iterative algorithm, since upper and lower edges of the iris are easily covered by eyelid and eyelash, after a vertical edge information of the iris is acquired by using the Sobel vertical edge detection, central parameter information of the iris is obtained through the ellipse fitting.

3. The gaze estimation method for the headset device based on the iris and the pupil according to claim 1 , wherein the step (2) includes:
a. calibration of an intrinsic parameter of the camera, wherein Zhang's plane calibration is used, and the intrinsic parameter of the camera is calibrated through a checkerboard pattern at different angles and distances shot by the camera at a fixed position, to acquire parameter information of a zoom factor in the conversion formula from a 2D image coordinate system of an object to the camera coordinate system;
b. a gaze screen is divided into four regions with the same size and the center of each region is provided with one calibration point, by substituting a 2D central coordinate of the pupil and a known depth information value of the pupil, i.e. a distance from a 3D center of the pupil to an optical center of the camera, into the formula in step a, the 3D coordinate parameter of the pupil is obtained;
c. according to a geometric similarity between the gaze direction of the human eyes in an iris plane and the gaze in a projection of the screen, a depth information value of the iris is calculated by using the coordinate of the calibration point on the screen and the 3D coordinate of the pupil acquired in step b, and then by substituting a 2D central coordinate of the iris and the depth information value of the iris into the formula in step a, the 3D coordinate parameter of the iris is obtained.

4. The gaze estimation method for the headset device based on the iris and the pupil according to claim 1 , wherein the step (3) includes:
a. when the human eyes gaze at any point on the screen, the extracted 2D central coordinate of the pupil is compared with the 2D center of the pupil which corresponds to four calibration points, to position the gaze region of the human eyes;
b. according to a characteristic that the pupil center and the iris center are non-concurrent in the 3D space leading to that the pupil center and the iris center are non-collinear in the 2D plan; the angle information of the human eye gaze estimation is acquired by using the 2D central coordinates of the pupil and iris;
c. the 3D coordinates of the pupil center and the iris center within the gaze region are calculated by using the 3D gaze mapping model established in step (2), the pupil-iris 3D gaze direction vector is obtained and combined with the angle information of the gaze estimation to obtain the final gaze point position of the human eyes.

Assignees

• Univ South China Tech

Inventors

• Qin Hubiao
• Fei Shu

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06K9/0061

  Physics · mapped topic

• G06K9/00604

  Physics · mapped topic

• G06V40/193

  Preprocessing; Feature extraction · CPC title

• G06V40/19 Primary

  Sensors therefor · CPC title