Image sensor

Patent metadata

Field	Value
Publication number	US-10904429-B2
Application number	US-201716337148-A
Country	US
Kind code	B2
Filing date	Oct 19, 2017
Priority date	Dec 7, 2016
Publication date	Jan 26, 2021
Grant date	Jan 26, 2021

Abstract

In an image sensor, high-speed recognition processing is performed using high-speed image data, which is different from low-speed image data used for displaying. In the image sensor, an imaging element captures an image of an object and generates frames of image data arranged in time series. A binarization processing unit performs binarization processing on each of the frames to generate binarized frames. A tracking processing unit generates a difference between binarized frames adjacent in time series and tracks a change in a position of the object included in the binarized frames.

First claim

The invention claimed is:

1. An image sensor, comprising:
an imaging element configured to:
capture an image of an object and;
generate frames of image data in time series based on the captured image;

a binarization processing unit configured to binarize each of the frames to generate binarized frames;
a tracking processing unit configured to:
generate a difference between the binarized frames adjacent in time series; and
track a change in a position of the object included in the binarized frames based on the difference, wherein
the imaging element captures an image of an eye of a living body as the object,
the binarization processing unit is further configured to detect a pupil in the captured image of the eye of the living body, and
the tracking processing unit is further configured to track line-of-sight in the captured image of the eye of the living body;

a moment generation unit configured to calculate an area of the pupil in the captured image of the eye of the living body; and
an aggregation processing unit configured to detect blinking based on the area of the pupil and a time during which the pupil is continuously recognized.

2. The image sensor according to claim 1 , further comprising
a filter processing unit configured to filter each of the frames, wherein the binarization processing unit is configured to binarize each of the filtered frames.

3. The image sensor according to claim 1 , wherein the moment generation unit is further configured to calculate a moment of the object included in the binarized frames based on a result of the tracking processing unit.

4. The image sensor according to claim 3 , further comprising a barycentric position generation unit configured to generate a barycentric position of the object included in the binarized frames based on the moment calculated by the moment generation unit.

5. The image sensor according to claim 4 , wherein the aggregation processing unit configured to execute an aggregation process related to the object based on the barycentric position of the object generated by the barycentric position generation unit.

6. The image sensor according to claim 5 , further comprising a display control unit configured to display a result of the aggregation process.

7. The image sensor according to claim 5 , wherein the aggregation processing unit is further configured to aggregate at least one of a number of the object, a size of the object, a movement rate of the object.

8. The image sensor according to claim 5 , wherein
the aggregation processing unit is further configured to detect a change in a shape of the object, and
the tracking processing unit is configured to track the object in a case where the change of the shape of the object is detected.

9. The image sensor according to claim 5 , wherein the aggregation processing unit is further configured to:
measure a speed of the object; and
calculate a statistical value of the speed of the object based on presence of a plurality of objects including the object.

10. The image sensor according to claim 1 , further comprising an interface configured to accept an operation input based on a manner of the blinking.

11. The image sensor according to claim 1 ,
further comprising a barycentric position generation unit configured to generate a barycentric position of the pupil in the captured image of the eye of the living body, wherein
the aggregation processing unit is further configured to determine a line-of-sight direction based on the barycentric position of the pupil.

12. The image sensor according to claim 11 , further comprising an interface configured to accept an operation input based on a manner of the blinking and the line-of-sight direction.

13. The image sensor according to claim 1 , wherein the aggregation processing unit is further configured to control the binarization processing unit to detect the pupil again in a case where the blinking is detected.

14. The image sensor according to claim 5 , wherein the aggregation processing unit further configured to:
measure a speed of the object; and
detect an abnormality in a case where the speed of the object does not satisfy a determined condition.

15. The image sensor according to claim 5 , wherein the aggregation processing unit is further configured to:
measure a vibration amount of the object; and
detect an abnormality in a case where the vibration amount of the object does not satisfy a determined condition.

16. The image sensor according to claim 5 , wherein the aggregation processing unit is further configured to:
measure a rotation amount of the object; and
detect an abnormality in a case where the rotation amount of the object does not satisfy a determined condition.

17. An image sensor, comprising:
an imaging element configured to:
capture an image of an object; and
generate frames of image data in time series based on the captured image;

a binarization processing unit configured to binarize each of the frames to generate binarized frames;
a tracking processing unit configured to:
generate a difference between the binarized frames adjacent in time series; and
track a change in a position of the object included in the binarized frames based on the difference;

a moment generation unit configured to calculate a moment of the object included in the binarized frames based on a result of the tracking processing unit;
a barycentric position generation unit configured to generate a barycentric position of the object included in the binarized frames based on the moment generated by the moment generation unit; and
an aggregation processing unit configured to execute aggregation processing related to the object based on the barycentric position of the object generated by the barycentric position generation unit, wherein
the imaging element captures an image of an eye of a living body as the object,
the binarization processing unit is further configured to detect a pupil in the captured image of the eye of the living body,
the tracking processing unit is further configured to track line-of-sight in the captured image of the eye of the living body,
the moment generation unit is further configured to calculate an area of the pupil in the captured image of the eye of the living body, and
the aggregation processing unit is further configured to detect blinking based on the area of the pupil and time during which the pupil is continuously recognized.

Assignees

• Sony Semiconductor Solutions Corp

Inventors

• Kobayashi Masatsugu
• Shida Sayaka
• Mitsunami Koki

Classifications

• G06V40/19

  Sensors therefor · CPC title

• G06V20/695

  Preprocessing, e.g. image segmentation · CPC title

• G06V10/507

  Summing image-intensity values; Histogram projection analysis · CPC title

• G06V10/30

  Noise filtering · CPC title

• G06V10/17

  using hand-held instruments · CPC title