Apparatus and method for robust eye/gaze tracking

Patent metadata

Field	Value
Publication number	US-9646207-B2
Application number	US-201514622594-A
Country	US
Kind code	B2
Filing date	Feb 13, 2015
Priority date	Feb 21, 2014
Publication date	May 9, 2017
Grant date	May 9, 2017

Abstract

At least one image registering unit records at least one series of images representing a subject. A control unit controls an operation sequence for the at least one image registering unit in such a manner that a subsequent data processing unit receives a repeating sequence of image frames there from, wherein each period contains at least one image frame of a first resolution and at least one image frame of a second resolution being different from the first resolution. Based on the registered image frames, the data processing unit produces eye/gaze tracking data with respect to the subject.

First claim

The invention claimed is:

1. An eye/gaze tracking apparatus comprising:
at least one image registering unit configured to record at least one series of images representing a subject;
a control unit configured to control an operation sequence for the at least one image registering unit; and
a data processing unit configured to receive an at least one predetermined period comprising a series of images from the at least one image registering unit, and based on the at least one predetermined period produce eye/gaze tracking data with respect to the subject,
wherein the control unit is configured to control the operation sequence for the at least one image registering unit such that the at least one predetermined period of series of images received by the data processing unit represents a repeating sequence of image frames containing at least one image frame of a first resolution and at least one image frame of a second resolution being different from the first resolution, wherein within the at least one predetermined period, the predetermined period includes a sequence of image frames having a first image frame of a first resolution followed in sequence by two or more second image frames of a second resolution.

2. The eye/gaze tracking apparatus according to claim 1 , wherein the at least one image registering unit comprises an image sensor having a set of sensor elements, and the at least one image registering unit is configured to deliver data registered by said sensor elements in the form of said image frames, where
the at least one image frame of the first resolution contains data from a first set of said sensor elements, and
the at least one image frame of the second resolution contains data from a second set of said sensor elements.

3. The eye/gaze tracking apparatus according to claim 2 , wherein said sensor elements are distributed over a sensor area having a full width and a full height, and the at least one image frame of the first resolution is an essentially full frame containing data from sensor elements located at positions which range over at least 80% of the full width and at least 80% of the full height of the sensor area.

4. The eye/gaze tracking apparatus according to claim 2 , wherein said sensor elements are distributed over a sensor area having a full width and a full height, and the at least one image frame of the second resolution is a sub-frame containing data from sensor elements located at positions which range from less than 50% of the full width and/or less than 50% of the full height of the sensor area.

5. The eye/gaze tracking apparatus according to claim 2 , wherein the first resolution is lower than the second resolution.

6. The eye/gaze tracking apparatus according to the claim 5 , comprising a first light source, and the control unit is configured to control the first light source in coordination with the operation sequence for the at least one image registering unit such that the first light source emits light while at least one of the at least one image frame of the second resolution is registered.

7. The eye/gaze tracking apparatus according to claim 6 , comprising a second light source, and the control unit is further configured to control the second light source in coordination with the operation sequence for the at least one image registering unit such that the second light source emits light while at least one image frame of the repeating sequence of image frames is registered.

8. The eye/gaze tracking apparatus according to claim 5 , wherein the at least one image registering unit is configured to produce an image frame of the first resolution by having each data point in the image frame representing combined data from at least two sensor elements in the image sensor.

9. The eye/gaze tracking apparatus according to claim 5 , wherein the at least one image registering unit is configured to produce an image frame of the first resolution by including data from less than all sensor elements in the image sensor in the image frame.

10. The eye/gaze tracking apparatus according to claim 5 , comprising at least one first image registering unit and at least one second image registering unit, wherein at least one of the at least one first image registering unit is configured to produce image frames of the first resolution and at least one of the at least one second image registering unit is configured to produce image frames of the second resolution.

11. The eye/gaze tracking apparatus according to claim 1 , comprising a single image registering unit configured to record a single series of images representing the subject, and the control unit is configured to control the operation sequence for the single image registering unit such that the at least one predetermined period contains at least one image frame of the first resolution followed in sequence by two or more second image frames of the second resolution.

12. The eye/gaze tracking apparatus according to claim 1 , wherein at least one image frame is analyzed to determine the presence of at least one of a facial gesture, hand gesture, facial feature, facial expression, eye closure, or head pose of a person imaged by the image frame.

13. The eye/gaze tracking apparatus according to claim 1 , wherein at least one image frame is analyzed to identify a user of the eye/gaze tracking apparatus.

14. The eye/gaze tracking apparatus according to claim 13 , wherein the at least one image frame being analyzed to identify a user of the eye/gaze tracking apparatus is of the first resolution.

15. A method of producing eye/gaze tracking data with respect to a subject, the method comprising:
registering, via at least one respective image registering unit, an at least one predetermined period comprising a series of images representing the subject;
processing the series of images from the at least one predetermined period;
producing the eye/gaze tracking data based on the at least one predetermined period; and
controlling an operation sequence for the at least one image registering unit such that the predetermined period of series of images to be processed represents a repeating sequence of image frames containing at least one image frame of a first resolution and at least one image frame of a second resolution being different from the first resolution, wherein within the at least one predetermined period, the predetermined period includes a sequence of image frames having a first image frame of a first resolution followed in sequence by two or more second image frames of a second resolution.

16. The method according to claim 15 , wherein the at least one image registering unit comprises an image sensor having a set of sensor elements, and the method comprises delivering data registered by said sensor elements in the form of said image frames, where
the at least one image frame of the first resolution contains data from a first set of said sensor elements; and
the at least one image frame of the second resolution contains data from a second set of said sensor element.

17. The method apparatus according to claim 16 , wherein said sensor elements are distributed over a sensor area having a full width and a full height, and the at least one image frame of the first resolution is an essentially full frame containing data from sensor elements located at positions which range over at least 80% of the full width and at least 80% of the full height of the sensor area.

18. The method according to claim 16 , wherein said sensor elements are distributed over a sensor area having a full width and a full h

Assignees

• Tobii Tech Ab
• Tobii Ab

Inventors

• Kuldkepp Mattias
• Skogo Marten
• Hanqvist Mattias
• Brogren Martin
• Muthusamy Dineshkumar

Classifications

• H04N23/74

  by influencing the scene brightness using illuminating means · CPC title

• H04N23/58

  Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors · CPC title

• H04N23/611 Primary

  where the recognised objects include parts of the human body · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title