Object removal from an image

Patent metadata

Field	Value
Publication number	US-9390532-B2
Application number	US-201214377045-A
Country	US
Kind code	B2
Filing date	Feb 7, 2012
Priority date	Feb 7, 2012
Publication date	Jul 12, 2016
Grant date	Jul 12, 2016

Abstract

A method including digitally processing a first image of a scene from a first perspective, to identify a first obscured portion of the scene at an imaging plane that is obscured by a forward object in the scene; digitally processing a second image of the scene from a second perspective, to identify a second obscured portion of the scene at the imaging plane that is obscured by the forward object in the scene; and outputting, if the first obscured portion of the scene and the second obscured portion of the scene are overlapping at the imaging plane, an indication that the first image and the second image cannot be processed successfully to produce a third composite image of the scene without the forward object.

First claim

I claim:

1. A method comprising:
digitally processing a first image of a scene from a first perspective, to identify a first obscured portion of the scene at an imaging plane that is obscured by a forward object in the scene;
digitally processing a second image of the scene from a second perspective, to identify a second obscured portion of the scene at the imaging plane that is obscured by the forward object in the scene;
determining if the first obscured portion of the scene and the second obscured portion of the scene are overlapping at the imaging plane;
and wherein if it is determined that the first obscured portion of the scene and the second obscured portion of the scene are overlapping at the imaging plane, not enabling the first image and second image to be processed to produce a third composite image of the scene, and outputting an indication that the first image and the second image cannot be processed to produce a third composite image of the scene without the forward object.

2. A method as claimed in claim 1 , wherein digitally processing the first image, digitally processing the second image and outputting the indication is performed as part of an image composition stage performed prior to capture and storage of the first image and the second image as retrievable data structures.

3. A method as claimed in claim 1 , further comprising digitally processing the first image and the second image to calculate a depth map for the scene.

4. A method as claimed in claim 1 , further comprising using a depth map to determine the object; or to determine the imaging plane or to determine if the first obscured portion of the scene and the second obscured portion of the scene are overlapping at the imaging plane, by using a relative size of the object in the imaged scene and distances of the object and the imaging plane.

5. A method as claimed in claim 1 , further comprising processing the first image and the second image to produce a third composite image of the scene without the forward object.

6. A method as claimed in claim 1 , comprising repeating the method for at least one different imaging plane.

7. A method as claimed in claim 1 , comprising repeating the method for at least one different forward object.

8. A computer program tangibly embodied on a record carrier that, when run on a computer, performs: the method of claim 1 .

9. An apparatus comprising:
at least one processor; and
at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:
digitally processing a first image of a scene from a first perspective, to identify a first obscured portion of the scene at an imaging plane that is obscured by a forward object in the scene;
digitally processing a second image of the scene from a second perspective, to identify a second obscured portion of the scene at the imaging plane that is obscured by the forward object in the scene;
determining if the first obscured portion of the scene and the second obscured portion of the scene are overlapping at the imaging plane;
and wherein if it is determined that the first obscured portion of the scene and the second obscured portion of the scene are overlapping at the imaging plane, not enabling the first image and second image to be processed to produce a third composite image of the scene, and outputting an indication that the first image and the second image cannot be processed to produce a third composite image of the scene without the forward object.

10. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
detecting a further object that lies partly within: at least both the first obscured portion and the second obscured portion; and the first obscured portion or the second obscured portion; and
positioning the imaging plane at the detected further object.

11. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
detecting a further object as overlapping an edge of the first obscured region and as overlapping an edge of the second obscured region; and
positioning the imaging plane at the detected further object.

12. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
automatically detecting an object that has a portion obscured by a further forward object in the first image and in the second image and that has a portion obscured by the further forward object in only one of the first image and the second image; and
automatically selecting the detected object as the forward object.

13. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
digitally processing the first image and the second image to calculate a depth map for the scene.

14. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
using a depth map to determine the object or using a depth map to determine if the first obscured portion of the scene and the second obscured portion of the scene are overlapping at the imaging plane, by using a relative size of the object in the imaged scene and distances of the object and the imaging plane.

15. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
if the first obscured portion and the second obscured portion of the background are non-overlapping, outputting an indication that the first image and the second image can be processed successfully to produce a third composite image of the scene without the forward object.

16. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
processing the first image and the second image to produce a third composite image of the scene with the forward object present but providing an indication that the forward object can be removed.

17. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
processing the first image and the second image to produce a third composite image of the scene without the forward object.

18. An apparatus as claimed in claim 9 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform:
digitally processing a first image of a scene from a first perspective, to identify a first obscured portion of the scene at a first imaging plane that is obscured by a first forward object in the scene;
digitally processing a second image of the scene from a second perspective, to identify a second obscured portion of the scene at the first imaging plane that is obscured by the first forward object in the scene; and
outputtin

Assignees

• Hyvarinen Olli Heikki Antero
• Nokia Technologies Oy

Inventors

• Hyvarinen Olli Heikki Antero

Classifications

• H04N23/635

  Region indicators; Field of view indicators · CPC title

• H04N5/272 Primary

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title