Technique for deblurring images

Patent metadata

Field	Value
Publication number	US-9767538-B2
Application number	US-201314018362-A
Country	US
Kind code	B2
Filing date	Sep 4, 2013
Priority date	Sep 4, 2013
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

An image capture application captures a sequence of images via a digital camera. The sequence of images may have undesirable levels of blurriness due to the motion of objects in the field of view of the digital camera or due to movement of the digital camera itself. A deblur engine within the image capture application generates image segments within one of the captured images, where a given image segment includes pixel values that move coherently between different images in the sequence. The deblur engine then deblurs each image segment based on the coherent motion of each different image segment and combines the resultant, deblurred image segments into a deblurred image. Advantageously, blurriness caused by the combined effects of moving objects and camera motion may be reduced, thereby improving the ability of a digital camera to provide high-quality images. As such, the user experience of digital photography may be enhanced.

First claim

The invention claimed is:

1. A computer-implemented method for generating an image, the method comprising:
capturing a sequence of images that includes a first image and a second image;
selecting a first image segment within the first image that represents a first group of real-world objects based on a comparison of the first image and the second image;
generating a first motion estimate for a first portion of the first image segment and a second motion estimate for a second portion of the first image segment relative to a region in the second image that corresponds to the first image segment;
performing a first deblurring operation on the first portion of the first image segment based on a first blur kernel derived from the first motion estimate to generate a first deblurred portion of the first image segment;
performing the first deblurring operation on the second portion of the first image segment based on a second blur kernel derived from the second motion estimate to generate a second deblurred portion of the first image segment; and
generating a deblurred version of the first image by combining the first deblurred portion of the first image segment and the second deblurred portion of the first image segment.

2. The computer-implemented method of claim 1 , wherein the comparison of the first image with the second image comprises identifying a first set of pixel values associated with the first image that moves coherently between the first image and the second image, wherein the first set of pixel values corresponds to the first group of real-world objects.

3. The computer-implemented method of claim 2 , wherein identifying the first set of pixel values associated with the first image comprises:
generating a velocity vector field representing the movements of each pixel value between the first image and the second image; and
grouping together a set of pixel values having velocity vectors of approximately equal magnitude and approximately equal direction to designate as the first set of pixel values.

4. The computer-implemented method of claim 1 , wherein performing the first deblurring operation comprises:
estimating the first blur kernel associated with the first portion of the first image segment based on the first motion estimate;
reducing blurriness within the first portion of the first image segment during exposure of the sequence of images based on the first blur kernel.

5. The computer implemented method of claim 4 , wherein estimating the first blur kernel comprises:
estimating the velocity of a first pixel value in a first set of pixel values associated with the first image segment relative to other pixel values in the first set of pixel values; and
determining an amount of blurriness associated with the first pixel value that is caused by the velocity of the first pixel value.

6. The computer-implemented method of claim 1 , further comprising:
generating a second image segment within the first image that represents a second group of real-world objects;
generating a third motion estimate for the second image segment relative to a region in the second image that corresponds to the second image segment;
performing a second deblurring operation based on the third motion estimate to generate a second deblurred image segment; and
superimposing the first deblurred portion of the first image segment, the second deblurred portion of the first image segment, and the second deblurred image segment to generate the deblurred version of the first image.

7. The computer-implemented method of claim 6 , wherein performing the second deblurring operation comprises:
estimating a third blur kernel associated with the second image segment based on the third motion estimate;
reducing blurriness within the second image segment during exposure of the sequence of images based on the third blur kernel.

8. The computer implemented method of claim 7 , wherein estimating the third blur kernel comprises:
estimating the velocity of a second pixel value in a second set of pixel values associated with the second image segment relative to other pixel values in the second set of pixel values; and
determining an amount of blurriness associated with the second pixel value that is caused by the velocity of the second pixel value.

9. The computer-implemented method of claim 1 , wherein the first image and the second image are sequential images.

10. The computer-implemented method of claim 1 , further comprising:
determining a first fraction of blur that is attributable to motion associated with the first portion of the first image segment; and
generating the first blur kernel based on the first fraction.

11. A non-transitory computer-readable medium storing program instructions that, when executed by a processing unit, cause the processing unit to generate an image by performing the steps of:
capturing a sequence of images that includes a first image and a second image;
selecting a first image segment within the first image that represents a first group of real-world objects based on a comparison of the first image and the second image;
selecting a second image segment within the first image that represents a second group of real-world objects based on a second comparison of the first image and the second image;
generating a first motion estimate for a first portion of the first image segment and a second motion estimate for a second portion of the first image segment relative to a region in the second image that corresponds to the first image segment;
generating a third motion estimate for a first portion of the second image segment and a fourth motion estimate for a second portion of the second image segment relative to a region in the second image that corresponds to the second image segment;
performing a first deblurring operation on the first portion of the first image segment based on a first blur kernel derived from the first motion estimate to generate a first deblurred portion of the first image segment;
performing the first deblurring operation on the second portion of the first image segment based on a second blur kernel derived from the second motion estimate to generate a second deblurred portion of the first image segment;
performing a second deblurring operation on the first portion of the second image segment based on a third blur kernel derived from the third motion estimate to generate a first deblurred portion of the second image segment;
performing the second deblurring operation on the second portion of the second image segment based on a fourth blur kernel derived from the fourth motion estimate to generate a second deblurred portion of the second image segment; and
combining the first deblurred portion of the first image segment, the second deblurred portion of the first image segment, the first deblurred portion of the second image segment, and the second deblurred portion of the second image segment to generate a deblurred version of the first image.

12. The non-transitory computer-readable medium of claim 11 , wherein the comparison of the first image with the second image comprises identifying a first set of pixel values associated with the first image that moves coherently between the first image and the second image, wherein the first set of pixel values corresponds to the first group of real-world objects.

13. The non-transitory computer-readable medium of claim 12 , wherein the step of identifying the first set of pixel values associated with the first image comprises:
generating a velocity vector field representing the movements of each pixel value between the first image and the second image; and
grouping

Assignees

• Nvidia Corp

Inventors

• Pulli Kari

Classifications

• G06T7/215

  Motion-based segmentation · CPC title

• G06T5/003 Primary

  Physics · mapped topic

• G06T2207/20201

  Motion blur correction · CPC title

• G06T5/73 Primary

  Deblurring; Sharpening · CPC title