Image sensor employing varied intra-frame analog binning

Patent metadata

Field	Value
Publication number	US-11570384-B2
Application number	US-202017011227-A
Country	US
Kind code	B2
Filing date	Sep 3, 2020
Priority date	Sep 3, 2020
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

A method performed with an image sensor having a pixel array. At least one frame of a scene may be obtained using the pixel array. At least one region of interest (ROI) is identified within the frame. Subsequent frames of the scene are obtained, which involves controlling the pixel array to perform high resolution imaging with respect to the at least one ROI and low resolution imaging using analog binning with respect to remaining regions of the frames.

First claim

What is claimed is:

1. A method performed with an image sensor having a pixel array, comprising:
obtaining at least one frame of a scene using the pixel array;
identifying at least one region of interest (ROI) within the at least one frame;
obtaining subsequent frames of the scene, which comprises controlling the pixel array to perform high resolution imaging with respect to the at least one ROI and low resolution imaging using analog binning with respect to remaining regions of the frames outside the at least one ROI;
providing image data obtained from the pixel array in pipelines according to the same resolution;
digitally processing each of the pipelines separately, which includes defect correction processing, noise reduction processing, and/or remosaicing processing, to provide a low resolution group of image data and at least one high resolution group of image data; and
stitching together the low resolution and high resolution groups of image data to form a unified frame to be displayed.

2. The method of claim 1 , further comprising:
forming a picture in picture (PIP) frame using the remaining regions represented in the unified frame as background image elements and at least one high resolution window corresponding to the at least one high resolution group of image data as a foreground image.

3. The method of claim 2 , further comprising enlarging the at least one high resolution window relative to its original size in relation to the background image elements.

4. The method of claim 1 , wherein the identifying of at least one ROI is performed automatically through commands initiated by a processor executing program instructions to identify particular patterns of image elements as ROIs.

5. The method of claim 1 , wherein the identifying at least one ROI is performed through a command initiated by a user interface.

6. The method of claim 1 , wherein:
the at least one ROI comprises a first ROI and a second ROI;
a first analog binning factor is used for imaging by the pixel array with respect to the first ROI; and
a second, lower binning factor is used for imaging by the pixel array with respect to the second ROI.

7. The method of claim 6 , wherein the first ROI comprises a vehicle and the second ROI comprises a license plate of the vehicle.

8. The method of claim 6 , wherein the first ROI comprises a human body and the second ROI comprises a face of the human body.

9. The method of claim 1 , wherein the high resolution imaging with respect to the at least one ROI is performed without analog binning.

10. The method of claim 1 , wherein no analog binning is applied with respect to the at least one ROI and analog binning factors of two or higher are applied with respect to the remaining regions of the frames.

11. The method of claim 1 , wherein:
the at least one ROI comprises a first ROI and a second ROI separated from the first ROI by a background region; and
an identical high resolution binning factor is used for imaging by the pixel array with respect to each of the first and second ROIs.

12. The method of claim 1 , wherein said digitally processing each of the pipelines separately comprises the defect correction processing.

13. The method of claim 1 , wherein the analog binning comprises:
defining a pixel block of adjacent pixels; and
at least one of (i) summing; (ii) averaging; and (iii) weighting charges from the pixels of the pixel block; and/or (iv) skipping pixel data of some of the adjacent pixels of the pixel block.

14. The method of claim 13 , wherein the digital binning is performed in an analogous manner as the analog binning using the at least one of the (i) summing; (ii) averaging; (ii) weighting; and (iv) skipping.

15. The method of claim 1 , wherein said digitally processing each of the pipelines separately includes the remosaicing processing.

16. The method of claim 1 , wherein said digitally processing each of the pipelines separately includes the noise reduction processing.

17. A method performed with an image sensor having a pixel array, comprising:
obtaining at least one frame of a scene using the pixel array;
identifying at least one region of interest (ROI) within the at least one frame;
obtaining subsequent frames of the scene, which comprises controlling the pixel array to perform high resolution imaging with respect to the at least one ROI and low resolution imaging using analog binning with respect to remaining regions of the frames outside the at least one ROI;
providing image data obtained from the pixel array in pipelines according to the same resolution;
digitally processing each of the pipelines separately to provide a low resolution group of image data and at least one high resolution group of image data;
stitching together the low resolution and high resolution groups of imaging data to form a unified frame to be displayed; and
prior to stitching together the low resolution and high resolution groups of image data, digitally binning the at least one high resolution group of image data at a binning factor sufficient for matching the low resolution imaging based on the analog binning;
wherein the unified frame is stitched together to form a display frame at the same resolution throughout.

18. An image sensor comprising:
a pixel array; and
at least one image processor executing instructions to:
obtain at least one frame of image data of a scene read out from the pixel array;
obtain subsequent frames of the scene, which comprises controlling the pixel array to perform high resolution imaging with respect to at least one region of interest (ROI) of the scene, and low resolution imaging using analog binning with respect to remaining regions of the frames outside the at least one ROI; and
digitally bin the at least one high resolution group of image data at a binning factor sufficient for matching the low resolution imaging based on the analog binning, to provide digitally binned image data.

19. The image sensor of claim 18 , further comprising:
providing image data obtained from the pixel array in pipelines according to the same resolution; and
digitally process each of the pipelines separately to provide a low resolution group of image data and at least one high resolution group of image data, and
wherein the at least one image processor further executes instructions to identify and track the at least one ROI.

20. The image sensor of claim 18 , wherein the at least one image processor further executes instructions to receive boundary information of the at least one ROI from a processor external to the image sensor.

21. The image sensor of claim 18 , wherein the at least one image processor further executes instructions to:
stitch together the low resolution group of image data and the digitally binned image data to form a unified frame to be displayed; and
generate a picture in picture (PIP) frame using the remaining regions represented in the unified frame as background image elements and at least one high resolution window corresponding to the at least one high resolution group of image data as a foreground image.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Duenyas Shahaf
• Yaffe Yoel
• Horowitz Guy
• Eisenberg Amit
• Hamami Shy
• Monzon Oded
• Bitan Gal
• Piepsh Yoav

Classifications

• H04N5/343 Primary

  Electricity · mapped topic

• H04N5/347

  Electricity · mapped topic

• H04N25/40

  Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled · CPC title

• H04N25/46 Primary

  by combining or binning pixels · CPC title

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title