Device, method, and storage medium for high dynamic range imaging of a combined image having a moving object

Patent metadata

Field	Value
Publication number	US-9251573-B2
Application number	US-201213596237-A
Country	US
Kind code	B2
Filing date	Aug 28, 2012
Priority date	Nov 28, 2011
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

A device that combines a first image photographed with a first amount of exposure and a second image photographed with a second amount of exposure lower than the first amount of exposure, thereby generating a combined image having a wider dynamic range than a dynamic range for the amounts of exposure of the first and second images, the device includes a motion region extraction unit configured to extract at least one motion region in which an object moving between the first image and the second image is shown; a combining ratio determining unit configured to increase a combining ratio of the second image to the first image for a pixel in a background region outside of the at least one motion region such that the higher a luminance value of a pixel of the first image, the higher the combining ratio.

First claim

What is claimed is:

1. A device that combines a first image photographed with a first amount of exposure and a second image photographed with a second amount of exposure lower than the first amount of exposure, thereby generating a combined image having a wider dynamic range than a dynamic range for the first amount of exposure of the first image and the second amount of exposure of the second image, the device comprising:
a processor; and
a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to execute:
extracting at least one motion region in which an object moving between the first image and the second image is shown;
adjusting a combining ratio of the second image to the first image for a pixel in a background region outside of the at least one motion region, the combining ratio being adjusted such that the higher a luminance value of a pixel of the first image, the higher the combining ratio, and the combining ratio at a given luminance value is set such that the higher a ratio of a noise component to a signal component of a pixel of the at least one motion region in the second image, the higher the combining ratio at the given luminance value;
determining the luminance value of a pixel of the combined image in the at least one motion region on the basis of a luminance value of a corresponding pixel of the second image, and determining a luminance value of a pixel of the combined image in the background region as a value obtained by combining luminance values of corresponding pixels of the first image and the second image in accordance with the combining ratio, thereby generating the combined image; and
obtaining a representative luminance value for each of the at least one motion region in any of the first image and the second image, the representative luminance value being representative of luminance values in each of the at least one motion region;

wherein the obtaining of the representative luminance value sets the combining ratio at the given luminance value for the pixel in the background region such that the smaller a statistical luminance value that is a statistical central value for the representative luminance value of each of the at least one motion region, the higher the combining ratio at the given luminance value.

2. The device according to claim 1 ,
wherein the adjusting sets the combining ratio at the given luminance value such that the higher a photographic sensitivity of an image sensor that has generated the first image and the second image when the first image and the second image have been generated, the higher the combining ratio at the given luminance value.

3. The device according to claim 1 ,
wherein a plurality of motion regions including the at least one motion region are extracted, and
a motion region, among the plurality of motion regions, having a representative luminance value is less than a combining limit value that is a lower limit of representative luminance values for which image quality of the combined image is allowed is deleted.

4. The device according to claim 1 ,
wherein a combining limit value for the combining ratio is set such that the higher a photographic sensitivity of an image sensor that has generated the first image and the second image when the first image and the second image have been generated, the lower the combining limit value.

5. A method that combines a first image photographed with a first amount of exposure and a second image photographed with a second amount of exposure lower than the first amount of exposure, thereby generating a combined image having a wider dynamic range than a dynamic range for the first amount of exposure of the first image and the second amount of exposure of the second image, the method comprising:
extracting, by a processor, at least one motion region in which an object moving between the first image and the second image is shown;
adjusting a combining ratio of the second image to the first image for a pixel in a background region outside of the at least one motion region, the combining ratio being adjusted such that the higher a luminance value of a pixel of the first image, the higher the combining ratio, and the combining ratio at a given luminance value is set such that the higher a ratio of a noise component to a signal component of a pixel of the at least one motion region in the second image, the higher the combining ratio at the given luminance value;
determining the luminance value of a pixel of the combined image in the at least one motion region on the basis of a luminance value of a corresponding pixel of the second image, and determining a luminance value of a pixel of the combined image in the background region as a value obtained by combining luminance values of corresponding pixels of the first image and the second image in accordance with the combining ratio, thereby generating the combined image; and
obtaining a representative luminance value for each of the at least one motion region in any of the first image and the second image, the representative luminance value being representative of luminance values in each of the at least one motion region;
wherein the obtaining of the representative luminance value sets the combining ratio at the given luminance value for the pixel in the background region such that the smaller a statistical luminance value that is a statistical central value for the representative luminance value of each of the at least one motion region, the higher the combining ratio at the given luminance value.

6. The method according to claim 5 ,
wherein the combining ratio at the given luminance value is set such that the higher a photographic sensitivity of an image sensor that has generated the first image and the second image when the first image and the second image have been generated, the higher the combining ratio at the given luminance value.

7. The method according to claim 5 ,
wherein a plurality of motion regions including the at least one motion region are extracted, and
a motion region, among the plurality of motion regions, whose representative luminance value is less than a combining limit value that is a lower limit of representative luminance values for which image quality of the combined image is allowed is deleted, even with use of a luminance value of a pixel in the motion region of the second image for generation of the combined image.

8. The method according to claim 5 ,
wherein a combining limit value for the combining ratio is set such that the higher a photographic sensitivity of an image sensor that has generated the first image and the second image when the first image and the second image have been generated, the lower the combining limit value.

9. A non-transitory computer-readable storage medium storing an image combining program that combines a first image photographed with a first amount of exposure and a second image photographed with a second amount of exposure lower than the first amount of exposure, thereby generating a combined image having a wider dynamic range than a dynamic range for the first amount of exposure of the first image and the second amount of exposure of the second image, the image combining program causing a computer to execute a process comprising:
extracting, by a processor, at least one motion region in which an object moving between the first image and the second image is shown;
adjusting a combining ratio of the second image to the first image for a pixel in a background region outside of the at least one motion region, the combining ratio being adjusted such that the higher a luminance value of a pixel of the first image, the higher the combining ratio, and the combining ratio at a gi

Assignees

• Kobayashi Hiroyuki
• Fujitsu Ltd

Inventors

• Kobayashi Hiroyuki

Classifications

• G06T5/009 Primary

  Physics · mapped topic

• G06T7/2006

  Physics · mapped topic

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• G06T7/215

  Motion-based segmentation · CPC title