Image enhancement

Patent metadata

Field	Value
Publication number	US-9286658-B2
Application number	US-201213427167-A
Country	US
Kind code	B2
Filing date	Mar 22, 2012
Priority date	Mar 22, 2012
Publication date	Mar 15, 2016
Grant date	Mar 15, 2016

Abstract

Techniques described in the disclosure are generally related to enhancing portions of an image relative to other portions of the image. The example techniques may utilize depth information in conjunction with one or more viewer perceivable information to enhance portions of the image relative to other portions. The techniques may then display the enhanced image to provide the viewer with a possibly more realistic image.

First claim

The invention claimed is:

1. A method for image processing comprising:
receiving a constructed image that includes a first portion and a second portion;
determining depth map values for a depth map for the constructed image that indicates relative depth of pixels within the constructed image by at least determining which pixels are for objects in a foreground of the constructed image;
determining perception map values for a perception map for the constructed image by at least determining motion information of pixels of objects in the constructed image; and
enhancing the first portion of the constructed image relative to the second portion of the constructed image based on at least the depth map values and the perception map values to generate an enhanced version of the constructed image, wherein the first portion of the constructed image includes pixels for objects that are in the foreground of the constructed image and that are moving slower than a threshold, as determined by the perception map values, and wherein the second portion of the constructed image includes pixels of objects that are in a background of the constructed image and pixels of objects that are moving faster than the threshold, as determined by the perception map values.

2. The method of claim 1 , wherein enhancing the first portion of the constructed image relative to the second portion of the constructed image comprises at least one of:
enhancing pixels within the first portion, and not enhancing pixels within the second portion;
blurring the pixels within the second portion, and not blurring the pixels within the first portion; and
enhancing the pixels within the first portion, and blurring the pixels within the second portion.

3. The method of claim 1 ,
wherein determining perception map values comprises determining saliency information for pixels in the constructed image, and
wherein enhancing the first portion of the constructed image relative to the second portion comprises enhancing pixels that are more salient and for objects that are in the foreground of the constructed image relative to pixels that are less salient and for objects that are in a background of the constructed image.

4. The method of claim 1 , wherein the perception map comprises a motion information map indicative of perspective speed of objects in the constructed image, or the motion information map indicative of perspective speed of objects in the constructed image and a saliency map indicative of saliency of pixels in the constructed image.

5. The method of claim 1 , wherein the constructed image comprises a first constructed image of stereoscopic video for a first eye, the method further comprising:
receiving a second constructed image of the stereoscopic video for a second eye, wherein image content of the first constructed image and the second constructed image is substantially similar; and
enhancing a first portion of the second constructed image relative to a second portion of the second constructed image based on at least the depth map values for the first constructed image and the perception map values for the first constructed image.

6. The method of claim 1 , further comprising:
performing a weighted sum on the depth map values and the perception map values to generate enhancement map values for an enhancement map,
wherein enhancing the first portion of the constructed image relative to the second portion of the constructed image comprises enhancing the first portion of the constructed image relative to the second portion of the constructed image based on the enhancement map values.

7. The method of claim 1 , further comprising, at least one of:
filtering the depth values of the depth map;
smoothing the depth values of the depth map; and
histogram stretching the depth values of the depth map.

8. The method of claim 1 , wherein the constructed image and the enhanced version of the constructed image are from the same view.

9. A device comprising:
a memory configured to store a constructed image that includes a first portion and a second portion; and
one or more processing units configured to:
receive the constructed image;
determine depth map values for a depth map for the constructed image that indicates relative depth of pixels within the constructed image by at least determining which pixels are for objects in a foreground of the constructed image;
determine perception map values for a perception map for the constructed image by at least determining motion information of objects in the constructed image; and
enhance the first portion of the constructed image relative to the second portion of the constructed image based on at least the depth map values and the perception map values to generate an enhanced version of the constructed image, wherein the first portion of the constructed image includes pixels for objects that are in the foreground of the constructed image and that are moving slower than a threshold, as determined by the perception map values, and wherein the second portion of the constructed image includes pixels of objects that are in a background of the constructed image and pixels of objects that are moving faster than the threshold, as determined by the perception map values.

10. The device of claim 9 , wherein to enhance the first portion of the constructed image relative to the second portion, the one or more processing units are configured to, at least one of:
enhance pixels within the first portion, and not enhance pixels within the second portion;
blur the pixels within the second portion, and not blur the pixels within the first portion; and
enhance the pixels within the first portion, and blur the pixels within the second portion.

11. The device of claim 9 , wherein the one or more processing units are configured to:
determine perception map values for the perception map by at least determining saliency information for pixels in the constructed image, and
enhance the first portion of the constructed image relative to the second portion by at least enhancing pixels that are more salient and for objects that are in the foreground of the constructed image relative to pixels that are less salient and for objects that are in a background of the constructed image.

12. The device of claim 9 , wherein the perception map comprises a motion information map indicative of perspective speed of objects in the constructed image, or the motion information map indicative of perspective speed of objects in the constructed image and a saliency map indicative of saliency of pixels in the constructed image.

13. The device of claim 9 , wherein the constructed image comprises a first constructed image of stereoscopic video for a first eye, wherein the one or more processing units are configured to:
receive a second constructed image of the stereoscopic video for a second eye, wherein image content of the first constructed image and the second constructed image is substantially similar; and
enhance a first portion of the second constructed image relative to a second portion of the second constructed image based on at least the depth map values for the first constructed image and the perception map values for the first constructed image.

14. The device of claim 9 , wherein the one or more processing units are configured to:
perform a weighted sum on the depth map values and the perception map values to generate enhancement map values for an enhancement map; and
enhance the first portion of the constructed image relative to the second portion of the constructed image based on the enhancement map values.

15. The device of

Assignees

• Bhaskaran Vasudev
• Qualcomm Inc

Inventors

• Bhaskaran Vasudev

Classifications

• G06T5/002 Primary

  Physics · mapped topic

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/20012

  Locally adaptive · CPC title

• G06T5/70 Primary

  Denoising; Smoothing · CPC title