Image processing device, method of controlling image processing device and program causing computer to execute method

Patent metadata

Field	Value
Publication number	US-9609308-B2
Application number	US-201213549954-A
Country	US
Kind code	B2
Filing date	Jul 16, 2012
Priority date	Aug 24, 2011
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

There is provided an image processing device including an image acquisition part acquiring an image; a depth acquisition part acquiring a depth associated with a pixel in the image; a depth conversion part converting the depth in accordance with a function having a characteristic to nonlinearly approach a predetermined value with an increase in the depth; and a storage part storing the converted depth in association with the image.

First claim

What is claimed is:

1. An image processing device comprising:
a storage device; and
circuitry, coupled with the storage device, configured to
convert, in accordance with a predetermined function with a characteristic that varies depending on a coefficient, depth associated with at least one of a plurality of pixels within image data to have a nonlinear characteristic with an increase in the depth, a value of the coefficient depending on a shooting mode selected under which the image data is captured,
generate a depth map by associating aggregated depth data consisting of a plurality of converted depth data with the image data, each of the plurality of converted depth data being associated with one of the plurality of pixels within the image data, and
perform smoothing processing on the image data to a degree depending on the converted depth.

2. The image processing device according to claim 1 , wherein the circuitry converts the depth in accordance with the function having the nonlinear characteristic to nonlinearly approach a predetermined value with the increase in the depth, and
the circuitry performs the smoothing processing based on the depth converted in accordance with the characteristic.

3. The image processing device according to claim 2 , wherein the function is an exponential function defined by the following formula:
Y=α×e^(−βx), where x is the depth, y is an output, e is a base of natural logarithm, α is a predetermined constant and β is the coefficient.

4. The image processing device according to claim 2 , wherein the function is an exponential function defined by the following formula:
Y=α×{1−e^(−βx)}, where x is the depth, y is an output, e is a base of natural logarithm, α is a predetermined constant and β is the coefficient.

5. The image processing device according to claim 2 , wherein the circuitry is further configured to supply the value of the coefficient depending on the shooting mode under which the image data is captured.

6. The image processing device according to claim 5 , wherein the storage device is configured to store the set shooting mode in association with the image data, and
the circuitry supplies the coefficient depending on the stored set shooting condition.

7. The image processing device according to claim 1 , wherein the circuitry is further configured to create an aggregation of pixels as a depth image, each pixel having the converted depth value as a pixel value.

8. The image processing device according to claim 7 , wherein the circuitry is further configured to compress the depth image in accordance with a predetermined image compression format, and
the storage device is configured to store the compressed depth image in association with the image data.

9. A method of controlling an image processing device, the method comprising:
converting, with circuitry, in accordance with a predetermined function with a characteristic that varies depending on a coefficient, depth associated with at least one of a plurality of pixels within image data to have a nonlinear characteristic with an increase in the depth, a value of the coefficient depending on a shooting mode selected under which the image data is captured;
generating a depth map by associating aggregated depth data consisting of a plurality of converted depth data with the image data, each of the plurality of converted depth data being associated with one of the plurality of pixels within the image data; and
performing smoothing processing on the image data to a degree depending on the converted depth.

10. A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method, the method comprising:
converting, in accordance with a predetermined function with a characteristic that varies depending on a coefficient, depth associated with at least one of a plurality of pixels within image data to have a nonlinear characteristic with an increase in the depth, a value of the coefficient depending on a shooting mode selected under which the image data is captured;
generating a depth map by associating aggregated depth data consisting of a plurality of converted depth data with the image data, each of the plurality of converted depth data being associated with one of the plurality of pixels within the image data; and
performing smoothing processing on the image data to a degree depending on the converted depth.

11. The method according to claim 9 , wherein the converting converts the depth in accordance with the function having the nonlinear characteristic to nonlinearly approach a predetermined value with the increase in the depth, and
the performing performs the smoothing processing based on the depth converted in accordance with the characteristic.

12. The method according to claim 9 , further comprising:
creating an aggregation of pixels as a depth image, each pixel having the converted depth value as a pixel value.

13. The method according to claim 12 , further comprising:
compressing the depth image in accordance with a predetermined image compression format; and
storing the compressed depth image in association with the image data.

14. The non-transitory computer-readable storage medium according to claim 10 , wherein the converting converts the depth in accordance with the function having the nonlinear characteristic to nonlinearly approach a predetermined value with the increase in the depth, and
the performing performs the smoothing processing based on the depth converted in accordance with the characteristic.

15. The non-transitory computer-readable storage medium according to claim 10 , further comprising:
creating an aggregation of pixels as a depth image, each pixel having the converted depth value as a pixel value.

16. The image processing device according to claim 1 , wherein the circuitry is further configured to adjust the depth so that emphasis degree of a perspective becomes low in a landscape shooting mode compared to a normal shooting mode, when a set shooting mode is the landscape shooting mode.

17. The image processing device according to claim 1 , wherein the circuitry is further configured to adjust the depth so that emphasis degree of a perspective becomes low in a normal shooting mode compared to a macro shooting mode, when a set shooting mode is the normal shooting mode.

18. The image processing device according to claim 1 , wherein
the circuitry is further configured to
set the shooting mode from among a plurality of shooting modes including at least one of a normal shooting mode, a landscape shooting mode, and a macro shooting mode, in response to an input, the nonlinear characteristic being different between the set shooting mode and other shooting modes of the plurality of shooting modes,

the storage device is configured to store the set shooting mode in association with the image data, and
the circuitry is further configured to supply the value of the coefficient depending on the stored set shooting mode under which the image data is captured.

19. The image processing device according to claim 1 , wherein the circuitry is further configured to
set the shooting mode from among a plurality of shooting modes including at least one of a normal shooting mode, a landscape shooting mode, and a macro shooting mode, in response to an input, the nonlinear characteristic being different between the set shooting mode and other shooting modes of the plurality of shooting modes, a

Assignees

• Suzuki Naho
• Nabesako Hideki
• Mizukura Takami
• Sony Corp

Inventors

• Suzuki Naho
• Nabesako Hideki
• Mizukura Takami

Classifications

• G06T2207/10004

  Still image; Photographic image · CPC title

• H04N13/271 Primary

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T5/002

  Physics · mapped topic

• H04N13/0271 Primary

  Electricity · mapped topic