Image sensor for vision based on human computer interaction

Patent metadata

Field	Value
Publication number	US-11089254-B2
Application number	US-202016796501-A
Country	US
Kind code	B2
Filing date	Feb 20, 2020
Priority date	Apr 7, 2016
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

Techniques for reducing a read out time and power consumption of an image sensor used for eye tracking are described. In an example, a position of an eye element in an active area of a sensor is determined. The eye element can be any of an eye, a pupil of the eye, an iris of the eye, or a glint at the eye. A region of interest (ROI) around the position of the eye is defined. The image sensor reads out pixels confined to the ROI, thereby generating an ROI image that shows the eye element.

First claim

What is claimed is:

1. A method for reducing image sensor readout time and power consumption, the method comprising:
detecting, from an image generated based on an image sensor, an eye element that comprises at least one of: an eye, a pupil of the eye, an iris of the eye, or a glint at the eye;
determining a position corresponding to the eye element in an active area of the image sensor, wherein the active area corresponds to pixels of the image sensor;
positioning a region of interest (ROI) within the active area, wherein the ROI is positioned based on the position corresponding to the eye element and is smaller than the active area; and
reading out only a subset of the pixels of the image sensor, wherein the subset corresponds to the ROI;
wherein the image is a DFS image that shows the eye and that is generated in a first image frame, wherein the ROI is positioned around the position corresponding to the eye, and further comprising:
generating an ROI image based on pixel values of the subset in a second image frame, the ROI image showing the eye;
detecting, from the ROI image, another position corresponding to the pupil;
repositioning the ROI around the other position corresponding to the pupil;
reading out only an updated subset of the pixels of the image sensor, wherein the updated subset corresponds to the repositioned ROI; and
generating another ROI image based on updated pixel values of the updated subset in a third image frame, the other ROI image showing the pupil.

2. The method of claim 1 , wherein the ROI image is at least one of a BP ROI image or a DP ROI image that shows the eye, and wherein the other ROI image is either another BP ROI image or another DP ROI image that shows the pupil.

3. The method of claim 2 , wherein repositioning the ROI comprises reducing a size of the ROI such that the other ROI image shows the pupil and only a portion of the eye.

4. An eye tracking system, comprising:
an image sensor;
a processor; and
a memory storing computer-readable instructions that, upon execution by the processor, configure the eye tracking system to:
detect, from an image generated based on the image sensor, an eye element that comprises at least one of: an eye, a pupil of the eye, an iris of the eye, or a glint at the eye;
determine a position corresponding to the eye element in an active area of the image sensor, wherein the active area corresponds to pixels of the image sensor;
position a region of interest (ROI) within the active area, wherein the ROI is positioned based on the position corresponding to the eye element and is smaller than the active area; and
read out only a subset of the pixels of the image sensor, wherein the subset corresponds to the ROI;
wherein the image is a DFS image that shows the eye and that is generated in a first image frame, wherein the ROI is positioned around the position corresponding to the eye, and further configured to:
generate an ROI image based on pixel values of the subset in a second image frame, the ROI image showing the eye;
detect, from the ROI image, another position corresponding to the pupil;
reposition the ROI around the other position corresponding to the pupil;
read out only an updated subset of the pixels of the image sensor, wherein the updated subset corresponds to the repositioned ROI; and
generate another ROI image based on updated pixel values of the updated subset in a third image frame, the other ROI image showing the pupil.

5. The eye tracking system of claim 4 , wherein the ROI image is at least one of a BP ROI image or a DP ROI image that shows the eye, and wherein the other ROI image is either another BP ROI image or another DP ROI image that shows the pupil.

6. The eye tracking system of claim 4 , wherein repositioning the ROI comprises reducing a size of the ROI such that the other ROI image shows the pupil and only a portion of the eye.

7. A non-transitory computer readable medium storing instructions that, upon execution on an eye tracking system, configure the eye tracking system to perform operations comprising:
detecting, from an image generated based on an image sensor, an eye element that comprises at least one of: an eye, a pupil of the eye, an iris of the eye, or a glint at the eye;
determining a position corresponding to the eye element in an active area of the image sensor, wherein the active area corresponds to pixels of the image sensor;
positioning a region of interest (ROI) within the active area, wherein the ROI is positioned based on the position corresponding to the eye element and is smaller than the active area; and
reading out only a subset of the pixels of the image sensor, wherein the subset corresponds to the ROI;
wherein the image is a DFS image that shows the eye and that is generated in a first image frame, wherein the ROI is positioned around the position corresponding to the eye, and further comprising:
generating an ROI image based on pixel values of the subset in a second image frame, the ROI image showing the eye;
detecting, from the ROI image, another position corresponding to the pupil;
repositioning the ROI around the other position corresponding to the pupil;
reading out only an updated subset of the pixels of the image sensor, wherein the updated subset corresponds to the repositioned ROI; and
generating another ROI image based on updated pixel values of the updated subset in a third image frame, the other ROI image showing the pupil.

Assignees

• Tobii Ab

Inventors

• Ivarsson Magnus
• Stoltz Per-Edvin
• Masko David
• Ollesson Niklas
• Skogö Mårten
• Blixt Peter
• Jönsson Henrik

Classifications

• G06V40/19 Primary

  Sensors therefor · CPC title

• H04N25/44

  by partially reading an SSIS array · CPC title

• H04N25/131

  including elements passing infrared wavelengths · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title