Method, apparatus and computer program product for motion deblurring of images

Patent metadata

Field	Value
Publication number	US-2017352136-A1
Application number	US-201515539744-A
Country	US
Kind code	A1
Filing date	Dec 1, 2015
Priority date	Dec 29, 2014
Publication date	Dec 7, 2017
Grant date	—

Abstract

In an example embodiment a method, apparatus and computer program product are provided. The method includes determining presence of at least one moving object in a scene based on two or more burst images corresponding to the scene captured by a first camera. One or more portions of the scene associated with the at least one moving object are identified, and, information related to the one or more portions is provided to a second camera. An image of the scene captured by the second camera second camera is received, where a pixel level shutter disposed in front of an image sensor of the second camera is programmed to periodically open and close, throughout a duration of said image capture, for pixels of the image sensor corresponding to the one or more portions of the scene. A deblurred image corresponding to the scene is generated based on the image.

First claim

1 - 48 . (canceled)

49 . A method comprising:
determining presence of at least one moving object in a scene based on two or more burst images corresponding to the scene captured by a first camera, the two or more burst images are associated with a first exposure time; identifying one or more portions of the scene associated with the at least one moving object if the at least one moving object is determined to be present in the scene; providing information related to the one or more portions of the scene to a second camera, the second camera configured to capture single exposure images associated with a second exposure time, the second exposure time being greater than the first exposure time; receiving an image of the scene captured by the second camera, wherein a pixel level shutter disposed in front of an image sensor of the second camera is programmed to periodically open and close, throughout a duration of said image capture, for pixels of the image sensor corresponding to the one or more portions of the scene; and generating a deblurred image corresponding to the scene based on the image.

50 . The method as claimed in claim 49 , wherein the information related to the one or more portions of the scene associated with the at least one moving object is provided substantially instantaneously to the second camera subsequent to the capturing of the two or more burst images.

51 . The method as claimed in claim 49 , wherein the pixels of the image sensor corresponding to the one or more portions of the scene are identified based on maximum permissible disparity between the first camera and the second camera.

52 . The method as claimed in claim 49 , further comprising:
performing deconvolution of the image to remove motion blur caused by the at least one moving object in the scene, wherein removal of the motion blur from the image generates the deblurred image.

53 . The method as claimed in claim 52 , wherein information related to a portion of the scene from among the one or more portions of the scene comprises a velocity estimate of a corresponding moving object from among the at least one moving object.

54 . The method as claimed in claim 49 , further comprising performing, upon determining an absence of the at least one moving object in the scene:
providing indication of the absence of the at least one moving object in the scene to the second camera; and receiving a final image of the scene captured by the second camera, wherein the pixel level shutter is programmed to stay open, for all pixels of the image sensor throughout the duration of the image capture.

55 . The method as claimed in claim 54 , further comprising:
receiving a plurality of burst images corresponding to the scene captured within a time period equal to the second exposure time by the first camera, the plurality of burst images comprising the two or more burst images; and generating a secondary image corresponding to the scene based on the plurality of burst images by utilizing one or more deghosting algorithms.

56 . An apparatus comprising:
at least one processor; and at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform: determine presence of at least one moving object in a scene based on two or more burst images corresponding to the scene captured by a first camera, the two or more burst images are associated with a first exposure time; identify one or more portions of the scene associated with the at least one moving object if the at least one moving object is determined to be present in the scene; provide information related to the one or more portions of the scene to a second camera, the second camera configured to capture single exposure images associated with a second exposure time, the second exposure time being greater than the first exposure time; receive an image of the scene captured by the second camera, wherein a pixel level shutter disposed in front of an image sensor of the second camera is programmed to periodically open and close, throughout a duration of said image capture, for pixels of the image sensor corresponding to the one or more portions of the scene; and
generate a deblurred image corresponding to the scene based on the image.

57 . The apparatus as claimed in claim 56 , wherein the information related to the one or more portions of the scene associated with the at least one moving object is provided substantially instantaneously to the second camera subsequent to the capturing of the two or more burst images.

58 . The apparatus as claimed in claim 56 , wherein the pixels of the image sensor corresponding to the one or more portions of the scene are identified based on maximum permissible disparity between the first camera and the second camera.

59 . The apparatus as claimed in claim 56 , wherein the apparatus is further caused, at least in part to:
perform deconvolution of the image to remove motion blur caused by the at least one moving object in the scene, wherein removal of the motion blur from the image generates the deblurred image.

60 . The apparatus as claimed in claim 59 , wherein information related to a portion of the scene from among the one or more portions of the scene comprises a velocity estimate of a corresponding moving object from among the at least one moving object.

61 . The apparatus as claimed in claim 56 , wherein the apparatus upon determining an absence of the at least one moving object in the scene is further caused, at least in part to:
provide indication of the absence of the at least one moving object in the scene to the second camera; and receive a final image of the scene captured by the second camera, wherein the pixel level shutter is programmed to stay open, for all pixels of the image sensor throughout the duration of the image capture.

62 . The apparatus as claimed in claim 61 , wherein the apparatus is further caused, at least in part to:
receive a plurality of burst images corresponding to the scene captured within a time period equal to the second exposure time by the first camera, the plurality of burst images comprising the two or more burst images; and generate a secondary image corresponding to the scene based on the plurality of burst images by utilizing one or more deghosting algorithms.

63 . A computer program product comprising at least one computer-readable storage medium, the computer-readable storage medium comprising a set of instructions, which, when executed by one or more processors, cause an apparatus to at least perform:
determine presence of at least one moving object in a scene based on two or more burst images corresponding to the scene captured by a first camera, the two or more burst images are associated with a first exposure time; identify one or more portions of the scene associated with the at least one moving object if the at least one moving object is determined to be present in the scene; provide information related to the one or more portions of the scene to a second camera, the second camera configured to capture single exposure images associated with a second exposure time, the second exposure time being greater than the first exposure time; receive an image of the scene captured by the second camera, wherein a pixel level shutter disposed in front of an image sensor of the second camera is programmed to periodically open and close, throughout a duration of said image capture, for pixels of the image sensor corresponding to the one or more portions o

Assignees

• Nokia Technologies Oy

Inventors

• Uliyar Mithun
• Govindarao Krishna
• Putraya Gururaj
• S V Basavaraja

Classifications

• G01B11/245

  using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence) · CPC title

• G06T7/292

  Multi-camera tracking · CPC title

• G06T2207/20201

  Motion blur correction · CPC title

• H04N5/211

  Ghost signal cancellation (H04N23/81 takes precedence) · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title