Electronic device for performing video HDR processing based on image data obtained by plurality of image sensors

Patent metadata

Field	Value
Publication number	US-11288782-B2
Application number	US-202016899162-A
Country	US
Kind code	B2
Filing date	Jun 11, 2020
Priority date	Sep 23, 2019
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

An electronic device performs HDR processing by using a plurality of image sensor blocks including a first image sensor block that photographs an object at a first field of view and a second image sensor block that photographs the object at a second field of view greater than the first field of view. An image signal processor generates first image data and second image data based on the photographed object. A main processor crops image data among image data of the second sensor that correspond to image data of the first sensor, and performs HDR processing based on the first image data and the cropped second image data.

First claim

What is claimed is:

1. An electronic device for processing a plurality of video frames, comprising:
a first image sensor block configured to generate a first data signal by photographing an object at a first field of view and to sequentially generate a third data signal by photographing to the object at the first field of view;
a second image sensor block configured to generate a second data signal by photographing the object at a second field of view greater than the first field of view and to sequentially generate a fourth data signal by photographing the object at the second field of view;
an image signal processor configured to generate first image data of a first resolution based on the first data signal, to generate second image data of a second resolution based on the second data signal, to generate third image data of a third resolution based on the third data signal, and to generate fourth image data of a fourth resolution based on the fourth data signal; and
a main processor configured to:
obtain first cropped image data from among the second image data that correspond to the first image data,
obtain second cropped image data from among the fourth image data that correspond to the first image data or the third image data, and
perform high dynamic range (HDR) processing based on the first image data, the third image data, the first cropped image data, and the second cropped image data, to generate an HDR processed image,

wherein the HDR processed image is a first video frame from among the plurality of video frames.

2. The electronic device of claim 1 , wherein, when the first resolution is greater than a resolution of the first cropped image data, the main processor is further configured to perform binning on the first image data to obtain binned image data.

3. The electronic device of claim 2 , wherein a resolution of the binned image data is equal to the resolution of the first cropped image data.

4. The electronic device of claim 1 , wherein, when the first resolution is smaller than a resolution of the first cropped image data, the main processor is further configured to perform binning on the first cropped image data cropped to obtain binned image data.

5. The electronic device of claim 4 , wherein the first resolution is equal to a resolution of the binned image data.

6. The electronic device of claim 1 , wherein, when the first resolution is smaller than a resolution of the first cropped image data, the main processor is further configured to perform upscaling on the first image data to obtain upscaled image data.

7. The electronic device of claim 6 , wherein a resolution of the upscaled image data is equal to the resolution of the first cropped image data.

8. The electronic device of claim 1 , wherein, when the first resolution and the third resolution are greater than a resolution of the first cropped image data and a resolution of the second cropped image data, the main processor is further configured to perform binning on the first image data and the third image data to obtain binned image data.

9. The electronic device of claim 1 , wherein the first image sensor block comprises a first pixel array of a first plurality of pixels,
wherein the second image sensor block comprises a second pixel array of a second plurality of pixels, and
wherein a second number of the second plurality of pixels is greater than a first number of the first plurality of pixels.

10. The electronic device of claim 1 , wherein a resolution of the first cropped image data is half or less of the second resolution.

11. An electronic device for processing a plurality of video frames, comprising:
a first image sensor block configured to photograph an object at a first field of view and to generate a first data signal corresponding to the object photographed at the first field of view;
a second image sensor block configured to photograph the object at a second field of view greater than the first field of view and to generate a second data signal corresponding to the object photographed at the second field of view;
a third image sensor block configured to photograph the object at a third field of view greater than the second field of view and to generate a third data signal corresponding to the object photographed at the third field of view;
an image signal processor configured to generate first image data of a first resolution based on the first data signal, to generate second image data of a second resolution based on the second data signal, and to generate third image data of a third resolution based on the third data signal; and
a main processor configured to:
perform high dynamic range (HDR) processing based on the first image data, the second image data, and the third image data, and, in a first zoom factor period,
obtain cropped image data from among the third image data that correspond to the second image data, and
perform HDR processing based on the second image data and the cropped image data, to generate an HDR processed image,
wherein the HDR processed image is a first video frame of the plurality of video frames.

12. The electronic device of claim 11 , wherein, when the second resolution is greater than a resolution of the cropped image data, the main processor is further configured to perform binning on the second image data to obtain binned image data.

13. The electronic device of claim 11 , wherein, in a second zoom factor period greater than the first zoom factor period, the main processor is further configured to obtain second cropped image data from among the second image data that correspond to the first image data, and perform the HDR processing based on the first image data and the second cropped image data.

14. The electronic device of claim 13 , wherein, when the first resolution is greater than a resolution of the second cropped image data, the main processor is further configured to perform binning on the first image data to obtain binned image data.

15. A method of processing a plurality of video frames by using a first image sensor block having a first field of view and a second image sensor block having a second field of view greater than the first field of view, the method comprising:
generating, by the first image sensor block, a first data signal by photographing an object at the first field of view, and sequentially generating a third data signal by photographing the object at the first field of view;
generating, by the second image sensor block, a second data signal by photographing the object photographed at the second field of view, and sequentially generating a fourth data signal by photographing the object at the second field of view;
generating first image data of a first resolution based on the first data signal;
generating second image data of a second resolution based on the second data signal;
generating third image data of a third resolution based on the third data signal;
generating fourth image data of a fourth resolution based on the fourth data signal;
generating first cropped image data from among the second image data that correspond to the first image data;
generating second cropped image data from among the fourth image data that correspond to the first image data or the third image data; and
performing high dynamic range (HDR) processing based on the first image data, the third image data, the first cropped image data, and the second cropped image data, to generate an HDR processed image,
wherein the HDR processed image is a first video frame of the plurality of video frames.

16. The method of claim 15 , further comprising:

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Su-Young
• Lee Jeongwook
• Kim Daejung
• Sung Hwansun
• Yang Hyunseok
• Yi Eunjik
• Lee Joohyoung
• JEONG Sooman

Classifications

• H04N25/57 Primary

  Control of the dynamic range · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title