Computational-complexity adaptive method and system for transferring low dynamic range image to high dynamic range image

Patent metadata

Field	Value
Publication number	US-2017193639-A1
Application number	US-201514984466-A
Country	US
Kind code	A1
Filing date	Dec 30, 2015
Priority date	Dec 30, 2015
Publication date	Jul 6, 2017
Grant date	—

Abstract

The present invention provides a method for converting a low dynamic range (LDR) image to a high dynamic range (HDR) image, including obtaining an input LDR image; and in an HDR image database, selecting one or more HDR images that match the input LDR image as candidate images. Further, the candidate images are warped according to a spatial correspondence between the candidate images and the input LDR image. The input LDR image is decomposed to an illumination component and a texture component. The illumination component and the texture component are reconstructed respectively based on the warped candidate images. The reconstructed illumination component and the reconstructed texture component are combined to produce an output HDR image.

First claim

What is claimed is:

1 . A method for converting a low dynamic range (LDR) image to a high dynamic range (HDR) image, comprising:
obtaining an input LDR image; in an HDR image database, selecting one or more HDR images that match the input LDR image as candidate images; warping the candidate images according to a spatial correspondence between the candidate images and the input LDR image; decomposing the input LDR image to an illumination component and a texture component; respectively reconstructing the illumination component and the texture component based on the warped candidate images; and combining the reconstructed illumination component and the reconstructed texture component to produce an output HDR image.

2 . The method according to claim 1 , wherein selecting one or more HDR images that matches the input LDR image as candidate images further comprises:
calculating one or more image features of the input LDR image, wherein the one or more image features are dynamic range independent; calculating the one or more features of the HDR images in the HDR image database; and comparing the calculated one or more features of the input LDR image and the calculated one or more features of the HDR images to find top K matching HDR images as the candidate images, wherein K is an integer.

3 . The method according to claim 1 , wherein warping the candidate images according to a spatial correspondence between the candidate images and the input LDR image further comprises:
degrading one of the candidate images to an LDR image; estimating a warping function through scale-invariant feature transform (SIFT) flow scene registration between the input LDR image and the degraded candidate image; and warping the degraded candidate image using the warping function through SIFT flow scene registration to achieve dense scene alignment.

4 . The method according to claim 3 , wherein:
provided that x denotes pixel index, I l denotes the input LDR image, and I i h denotes an i th candidate image,
an exposure time Δt is used when degrading one of the candidate images to an LDR image; and
the exposure time Δt of the i th candidate image is obtained by
Δ t i =median x ( I l ( x )/ I i h ( x )).

5 . The method according to claim 1 , wherein respectively reconstructing the illumination component and the texture component based on the warped candidate images further comprises:
solving a global optimization problem to recover illumination information in the over-exposure regions based on the warped candidate images with at least one of a data fidelity term, a spatial smooth term and an illumination prior term.

6 . The method according to claim 1 , wherein respectively reconstructing the illumination component and the texture component based on the warped candidate images further comprises:
computing an illumination prior term by a Gaussian kernel fitting to recover illumination information in each over-exposure region.

7 . The method according to claim 1 , wherein respectively reconstructing the illumination component and the texture component based on the warped candidate images further comprises:
for a pixel in the over-exposure regions, finding corresponding pixels in the warped candidate images; and sampling around the corresponding pixels to obtain a texton for texture synthesis of the pixel.

8 . The method according to claim 1 , wherein:
the HDR image database is stored on a cloud server; the input LDR image is obtained by a user device and uploaded to the cloud server; and the step of selecting one or more HDR images that matches the input LDR image as candidate images and the step of warping the candidate images according to a spatial correspondence between the candidate images and the input LDR image are performed on the cloud server.

9 . The method according to claim 8 , wherein:
the input LDR image is down-sampled before uploading to the cloud server; and a plurality of HDR images in the HDR image database are down-sampled with different scale factors and stored at multiple resolutions.

10 . The method according to claim 8 , wherein:
a texton bank is saved on both the user device and the cloud server; the cloud server identifies texton index of pixels in the warped candidate images and send to the user device; and the user device obtains the identified texton index of pixels in the warped candidate images for texture synthesis of corresponding pixels in the texture component.

11 . A system for converting a low dynamic range (LDR) image to a high dynamic range (HDR) image, comprising:
an HDR image database including a plurality of HDR images; and one or more processors configured to:
obtain an input LDR image;
select, from the HDR image database, one or more HDR images that match the input LDR image as candidate images;
warp the candidate images according to a spatial correspondence between the candidate images and the input LDR image;
decompose the input LDR image to an illumination component and a texture component;
respectively reconstruct the illumination component and the texture component based on the warped candidate images; and
combine the reconstructed illumination component and the reconstructed texture component to produce an output HDR image.

12 . The system according to claim 11 , wherein the one or more processor is further configured to:
calculate one or more image features of the input LDR image, wherein the one or more image features are dynamic range independent; calculate the one or more features of the HDR images in the HDR image database; and compare the calculated one or more features of the input LDR image and the calculated one or more features of the HDR images to find top K matching HDR images as the candidate images, wherein K is an integer.

13 . The system according to claim 11 , wherein when warping the candidate images according to a spatial correspondence between the candidate images and the input LDR image, the one or more processor is further configured to:
degrade one of the candidate images to an LDR image; estimate a warping function through scale-invariant feature transform (SIFT) flow scene registration between the input LDR image and the degraded candidate image; and warp the degraded candidate image using the warping function through SIFT flow scene registration.

14 . The system according to claim 13 , wherein:
provided that x denotes pixel index, I l denotes the input LDR image, and I i h denotes an i th candidate image,
an exposure time Δt is used when degrading one of the candidate images to an LDR image; and
the exposure time Δt of the i th candidate image is obtained by
Δ t i =median x ( I l ( x )/ I i h ( x )).

15 . The system according to claim 14 , wherein when reconstructing the illumination component, the one or more processor is further configured to:
solve a global optimization problem recover illumination information in the over-exposure regions based on the warped candidate images, wherein the global optimization problem includes at least one of a data fidelity term, a spatial smooth term and an illumination prior term.

16 . The system according to claim 14 , wherein the one or more processor is further configured to:
compute an illumination prior term by a Gaussian kernel fitting to recover illumination information in each over-exposure region.

17 . The system according to claim 14 , wherein when reconstructing the texture component, the one or more processor is further configured to:
for a

Assignees

• Tcl Res America Inc

Inventors

• Dai Qiqin
• Ren Xiaobo
• Wang Haohong

Classifications

• G06V10/763

  Non-hierarchical techniques, e.g. based on statistics of modelling distributions · CPC title

• G06V10/462 Primary

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• H04N25/57

  Control of the dynamic range · CPC title

• G06F18/23213

  with fixed number of clusters, e.g. K-means clustering · CPC title

• G06F18/23211

  with adaptive number of clusters · CPC title