Image processing system

Patent metadata

Field	Value
Publication number	US-2016292835-A1
Application number	US-201415036231-A
Country	US
Kind code	A1
Filing date	Oct 17, 2014
Priority date	Nov 14, 2013
Publication date	Oct 6, 2016
Grant date	—

Abstract

Provided is an information processing system configured to: receive a plurality of images; select a standard image; search, in extracting partial images for complementing a too bright region/too dark region in the standard image from a reference images, using correction images obtained by subjecting each of the images to multivalued processing for each division of luminance components, for regions having matching shapes of respective regions included in the correction image of the standard image and the correction images of the other images; and complement image portions corresponding to the too bright region and/or too dark region in the correction image of the standard image using image portions, which are matching regions and correspond to appropriately bright regions, to thereby generate a synthetic image. Each image is dividable into at least three values: a too bright region, an appropriately bright region, and a too dark region, and has a relationship in which a boundary luminance between the too bright region and the appropriately bright region aligns with a boundary luminance between the appropriately bright region and the too dark region in a darker image having an adjacency relationship therewith.

First claim

1 . An image processing system, comprising:
an image reception unit configured to receive a plurality of images having different luminance components as input images; a standard image selection unit configured to select a standard image from among the plurality of images; a region-to-be-complemented search unit configured to perform, in extracting partial images for complementing a too bright region and/or too dark region in the standard image using other images of the plurality of images as reference images from the reference images, processing of searching, using correction images obtained by subjecting each of the input images to multivalued processing for each division of the luminance components, for regions having matching shapes using shapes of dividable regions included in the correction image of the standard image and shapes of dividable regions included in the correction images of the other images; a complementary processing unit configured to subject image portions corresponding to the too bright region and/or too dark region in the correction image of the standard image to complementary processing using image portions, which are matching regions in the reference images and correspond to appropriately bright regions in each of the reference images, to thereby generate a synthetic image; and an output unit configured to output the generated synthetic image, wherein, as the input images, each image is dividable into at least three values: a bright region, an appropriately bright region, and a dark region, and an image group having a luminance component relationship in which a boundary luminance between the bright region and the appropriately bright region of luminance components in each image aligns with a boundary luminance between the appropriately bright region and the dark region in a darker image having an adjacency relationship therewith in terms of luminance is used.

2 . An image processing system according to claim 1 , wherein the standard image selection unit is configured to perform, in selecting the standard image, based on an area of the appropriately bright region in an entirety or a predetermined range of each image, processing of selecting one or a plurality of higher-order images having large areas as the standard image.

3 . An image processing system according to claim 1 , wherein the region-to-be-complemented search unit is configured to perform, in performing the search processing, using the correction images categorized into three divisions indicating a bright region, an appropriately bright region, and a dark region, matching processing on shapes of dividable regions including the appropriately bright region included in a correction image of an image having an adjacency relationship in terms of luminance with reference to shapes of the too bright region and/or too dark region included in the correction image of the standard image to extract the partial images in the reference images, which are used for complementing, from the reference images.

4 . An image processing system according to claim 1 , wherein, in repeatedly executing the search processing and the complementary processing using a plurality of reference images, the search processing and the complementary processing are repeated while complementing the standard image using a reference image having an adjacency relationship in terms of luminance with the standard image, and then filling the too bright region and/or too dark region in the standard image so that the complemented standard image is further complemented using a next reference image having a adjacency relationship in terms of luminance.

5 . An image processing system according to claim 1 , wherein an image content of an image portion, which is a region in the reference image that matches the standard image and corresponds to the appropriately bright region in each of the reference images, is referenced to search for regions that match all or a part of the remaining reference images.

6 . An image processing system according to claim 1 , wherein, in searching for a matching region between the reference images and the standard image, the following cost function (1) is satisfied to search for the matching region in the reference images:
E Ω ( p )= E shape ( p )+α E texture ( p )  (1),

where:
E Ω (p) is a total matching cost of individual luminance region shapes;
E shape (p) is a part of the cost function for searching with exclusive disjunction;
E texture (p) is a part of the cost function for searching with weighted SSD; and
α is a weight variable.

7 . An image processing system according to claim 1 , wherein, in the search processing, a plurality of extracted regions having a predetermined approximate relationship in which divisions of the luminance components are the same are treated as one or a plurality of groups.

8 . An image processing system according to claim 1 , wherein the plurality of images are received to receive the input images, and the input images are rearranged in order of luminance and thresholds with which the luminance component relationship is established are determined among the images so that the luminance component relationship among the input images is satisfied.

9 . An image processing method using an information processing system, including:
receiving a plurality of images having different luminance components as input images; selecting a standard image from among the plurality of images; performing, in extracting partial images for complementing a too bright region and/or too dark region in the standard image using other images in the plurality of images as reference images from the reference images, processing of searching, using correction images obtained by subjecting each of the input images to multivalued processing for each division of luminance components, for regions having matching shapes using shapes of dividable regions included in the correction image of the standard image and shapes of dividable regions included in the correction images of the other images; and subjecting image portions corresponding to the too bright region and/or too dark region in the correction image of the standard image to complementary processing using image portions, which are matching regions in the reference images and correspond to appropriately bright regions in each of the reference images, to thereby generate a synthetic image, wherein, as the input images, each image is dividable into at least three values: a too bright region, an appropriately bright region, and a too dark region, and an image group having a luminance component relationship in which a boundary luminance between the too bright region and the appropriately bright region of the luminance components in each image aligns with a boundary luminance between the appropriately bright region and the too dark region in a darker image having an adjacency relationship therewith in terms of luminance is used.

10 . A computer-readable recording medium having non-temporarily recorded thereon a program for causing an information processing system to operate as:
an image reception unit configured to receive a plurality of images having different luminance components as input images; a standard image selection unit configured to select a standard image from among the plurality of images; a region-to-be-complemented search unit configured to perform, in extracting partial images for complementing a too bright region and/or too dark region in the standard image using other images in the plurality of images as reference images from the reference images, processing of searching, using correction images obtained by subjecting each of the input images to multivalued processing f

Assignees

• Nec Corp
• Tokyo Inst Of Technnology
• Tokyo Inst Tech

Inventors

• Shibata Takashi
• Senda Shuji
• Tanaka Masayuki
• Okutomi Masatoshi
• Hayami Takehito

Classifications

• H04N5/265

  Mixing · CPC title

• G06T2207/10144

  Varying exposure · CPC title

• H04N1/4072

  dependent on the contents of the original · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title