Method and apparatus for processing array image

Patent metadata

Field	Value
Publication number	US-12389133-B2
Application number	US-202318376280-A
Country	US
Kind code	B2
Filing date	Oct 3, 2023
Priority date	Oct 21, 2022
Publication date	Aug 12, 2025
Grant date	Aug 12, 2025

Abstract

An image processing method includes: based on an imaging device generating an input array image of a multi-view through an array lens, determining a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view; determining a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation; determining an image processing mode based on the first setting parameter and the second setting parameter; and generating an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.

First claim

What is claimed is:

1. An image processing method comprising:
based on an imaging device generating an input array image of a multi-view through an array lens, determining a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view;
determining a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation;
determining an image processing mode based on the first setting parameter and the second setting parameter; and
generating an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.

2. The image processing method of claim 1 , wherein the determined image processing mode comprises:
a first mode that is determined when the first setting parameter indicates to use the multi-view and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a second mode that is determined when the first setting parameter indicates to use the multi-view and the second setting parameter indicates the priority is assigned to the sensitivity enhancement,
a fourth mode that is determined when the first setting parameter indicates to use the single view and the second setting parameter indicates the priority is assigned to the resolution enhancement, or
a fifth mode that is determined when the first setting parameter indicates to use the single view and the second setting parameter indicates the priority is assigned to the sensitivity enhancement.

3. The image processing method of claim 2 ,
wherein the processing procedure that is based on the first mode comprises sub stages, and
wherein the processing procedure that is based on the second mode, the fourth mode, or the fifth mode comprises only some of the sub stages.

4. The image processing method of claim 3 , wherein the sub stages comprise:
a first sub stage for performing demosaicing on raw data,
a second sub stage for performing upsampling for resolution enhancement,
a third sub stage for generating matching information of the multi-view, and
a fourth sub stage for generating a synthesized image using the matching information.

5. The image processing method of claim 4 ,
wherein the processing procedure that is based on the second mode does not include the first sub stage and the second sub stage,
wherein the processing procedure that is based on the fourth mode does not include the third sub stage and the fourth sub stage, and
wherein the processing procedure that is based on the fifth mode does not include the second sub stage, the third sub stage, and the fourth sub stage.

6. The image processing method of claim 1 , further comprising:
determining a third setting parameter indicating whether to use hardware logic-based binning processing,
wherein the determining the image processing mode is further based on the third setting parameter.

7. The image processing method of claim 6 , wherein the determined image processing mode comprises:
a first mode that is determined when the first setting parameter indicates to use the multi-view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a second mode that is determined when the first setting parameter indicates to use the multi-view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the sensitivity enhancement,
a third mode that is determined when the first setting parameter indicates to use the multi-view and the third setting parameter indicates to use the hardware logic-based binning processing,
a fourth mode that is determined when the first setting parameter indicates to use the single view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a fifth mode that is determined when the first setting parameter indicates to use the single view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the sensitivity enhancement, and
a sixth mode that is determined when the first setting parameter indicates to use the single view and the third setting parameter indicates to use the hardware logic-based binning processing.

8. The image processing method of claim 6 , wherein the third setting parameter indicates to use the hardware logic-based binning processing in at least one of a low-illuminance environment or a video capturing mode.

9. The image processing method of claim 6 , wherein the hardware logic-based binning processing comprises:
determining initial corresponding pixels between sub images that respectively correspond to different views of the input array image,
extracting a refinement target from the initial corresponding pixels based on an intensity difference between the initial corresponding pixels,
determining a new corresponding pixel for the refinement target by using a calibration parameter, and
performing binning on final corresponding pixels through a weighted average of the final corresponding pixels that are based on the new corresponding pixel.

10. The image processing method of claim 1 , wherein the first setting parameter indicates to use the single view in response to a demand for low power.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform an image processing method comprising:
Based on an imaging device generating an input array image of a multi-view through an array lens, determining a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view;
determining a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation;
determining an image processing mode based on the first setting parameter and the second setting parameter; and
generating an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.

12. An image processing apparatus comprising:
a processor; and
a memory configured to store instructions executable by the processor,
wherein, in response to the instructions being executed by the processor, the processor is configured to:
based on an imaging device generating an input array image of a multi-view through an array lens, determine a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view,
determine a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation,
determine an image processing mode based on the first setting parameter and the second setting parameter, and
generate an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.

13. The image processing apparatus of claim 12 , wherein the determined image processing mode comprises:
a first mode that is determined when the

Assignees

• Samsung Electronics Co Ltd

Inventors

• Heo Jingu
• Kang Byong Min
• Park Juyong
• Nam Dong Kyung
• Cho Yang Ho

Classifications

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/651

  for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera · CPC title

• H04N23/951 Primary

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N25/46

  by combining or binning pixels · CPC title