Depth-based image blurring

Patent metadata

Field	Value
Publication number	US-10552947-B2
Application number	US-201715824574-A
Country	US
Kind code	B2
Filing date	Nov 28, 2017
Priority date	Jun 26, 2012
Publication date	Feb 4, 2020
Grant date	Feb 4, 2020

Abstract

An image such as a light-field image may be processed to provide depth-based blurring. The image may be received in a data store. At an input device, first and second user input may be received to designate a first focus depth and a second focus depth different from the first focus depth, respectively. A processor may identify one or more foreground portions of the image that have one or more foreground portion depths, each of which is less than the first focus depth. The processor may also identify one or more background portions of the image that have one or more background portion depths, each of which is greater than the second focus depth. The processor may also apply blurring to the one or more foreground portions and the one or more background portions to generate a processed image, which may be displayed on a display device.

First claim

What is claimed is:

1. A method for processing an image, the method comprising:
in a data store, receiving the image;
on a display device, displaying a range bar comprising a first slider and a second slider;
in an input device, receiving a first user input that designates a first focus depth by detecting motion selecting to move the first slider along the range bar to a first location corresponding to the first focus depth;
in the input device, receiving a second user input that designates a second focus depth different from the first focus depth by detecting motion selecting to move the second slider along the range bar to a second location corresponding to the second focus depth;
in a processor, identifying one or more foreground portions of the image, wherein the one or more foreground portions have one or more foreground portion depths, each of which is less than the first focus depth;
in the processor, identifying one or more background portions of the image, wherein the one or more background portions have one or more background portion depths, each of which is greater than the second focus depth;
in the processor, applying blurring to the one or more foreground portions and to the one or more background portions to generate a processed image; and
on the display device, displaying the processed image.

2. The method of claim 1 , further comprising, in the data store, receiving a depth map indicative of a plurality of depths at which a plurality of objects were disposed, relative to a camera used to capture the image, during capture of the image;
wherein identifying the one or more foreground portions comprises determining, based on the depth map, that the one or more foreground portions have the one or more foreground portion depths;
and wherein identifying the one or more background portions comprises determining, based on the depth map, that the one or more background portions have the one or more background portion depths.

3. The method of claim 2 , wherein the image comprises a light-field image captured with a light-field image capture device.

4. The method of claim 1 , further comprising, on the display device:
displaying a graphical user interface comprising the first slider and the second slider;
in response to receipt of the first user input, displaying motion of the first slider to the first location; and
in response to receipt of the second user input, displaying motion of the second slider to the second location.

5. The method of claim 1 , wherein the first focus depth is less than the second focus depth such that one or more intermediate portions of the image have one or more intermediate portion depths that are greater than the first focus depth and less than the second focus depth;
and wherein generating the processed image comprises rendering the one or more intermediate portions of the image without blurring.

6. The method of claim 1 , wherein the first focus depth is greater than the second focus depth such one or more of the one or more foreground portions of the image are also included in the one or more background portions of the image;
and wherein generating the processed image comprises applying blurring to an entirety of the image.

7. The method of claim 1 , wherein applying the blurring to the one or more foreground portions and to the one or more background portions comprises applying a noncircular blur shape to the one or more foreground portions and to the one or more background portions.

8. The method of claim 1 , wherein applying the blurring to the one or more foreground portions and to the one or more background portions comprises:
applying a first noncircular blur shape to one or more first subsets of the one or more foreground portions and the one or more background portions that are at a first depth; and
applying a second noncircular blur shape to one or more second subsets of the one or more foreground portions and the one or more background portions that are at a second depth different from the first depth.

9. The method of claim 1 , wherein applying the blurring to the one or more foreground portions and to the one or more background portions comprises applying the blurring to each pixel of a plurality of pixels of the image based, at least partially, on at least one of an X coordinate of the pixel, and a Y coordinate of the pixel.

10. The method of claim 9 , further comprising, in the input device, prior to applying the blurring to the one or more foreground portions and to the one or more background portions, receiving a third user input that designates a tilt angle at which a focus plane of the image is to be tilted;
wherein applying the blurring to the one or more foreground portions and to the one or more background portions further comprises applying the blurring to portions of the image that are displaced from the focus plane.

11. The method of claim 10 , further comprising, in the input device, prior to applying the blurring to the one or more foreground portions and to the one or more background portions, receiving a fourth user input that designates a rotation angle at which the focus plane is to be rotated.

12. A non-transitory computer-readable medium for processing an image, comprising instructions stored thereon, that when executed by a processor, perform the steps of:
causing a data store to receive the image;
causing a display device to display a range bar comprising a first slider and a second slider;
causing an input device to receive a first user input that designates a first focus depth by detecting motion selecting to move the first slider along the range bar to a first location corresponding to the first focus depth;
causing the input device to receive a second user input that designates a second focus depth different from the first focus depth by detecting motion selecting to move the second slider along the range bar to a second location corresponding to the second focus depth;
identifying one or more foreground portions of the image, wherein the one or more foreground portions have one or more foreground portion depths, each of which is less than the first focus depth;
identifying one or more background portions of the image, wherein the one or more background portions have one or more background portion depths, each of which is greater than the second focus depth;
applying blurring to the one or more foreground portions and to the one or more background portions to generate a processed image; and
causing the display device to display the processed image.

13. The non-transitory computer-readable medium of claim 12 , further comprising instructions stored thereon, that when executed by a processor, cause the data store to receive a depth map indicative of a plurality of depths at which a plurality of objects were disposed, relative to a camera used to capture the image, during capture of the image;
wherein identifying the one or more foreground portions comprises determining, based on the depth map, that the one or more foreground portions have the one or more foreground portion depths;
and wherein identifying the one or more background portions comprises determining, based on the depth map, that the one or more background portions have the one or more background portion depths.

14. The non-transitory computer-readable medium of claim 12 , further comprising instructions stored thereon, that when executed by a processor, perform the steps of:
causing the display device, in response to receipt of the first user input, to display motion of the first slider to a first location corresponding to the first focus depth; and
causing the displ

Assignees

• Google Llc

Inventors

• Liang Chia-Kai
• Oberheu Kent
• Akeley Kurt
• Girod Garrett
• Karnad Nikhil
• Benevides Francis A

Classifications

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/20012

  Locally adaptive · CPC title