Scene motion correction in fused image systems

Patent metadata

Field	Value
Publication number	US-9681050-B2
Application number	US-201615154659-A
Country	US
Kind code	B2
Filing date	May 13, 2016
Priority date	May 30, 2014
Publication date	Jun 13, 2017
Grant date	Jun 13, 2017

Abstract

Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

First claim

The invention claimed is:

1. A method comprising:
obtaining a plurality of images from a burst capture, wherein the plurality of images includes at least one or more short-exposure images and at least one or more long-exposure images;
identifying a first group of images from the plurality of images, wherein the first group of images contains fewer images than the plurality of images;
analyzing the first group of images to identify a predetermined object, the predetermined object having a corresponding location;
weighting regions of at least some of the first group of images based, at least in part, on the corresponding location of the predetermined object;
combining at least two of the first group of images based, at least in part, on the weighted regions to generate a combined image; and
storing the combined image in a memory.

2. The method of claim 1 , wherein the first group of images comprises one or more short-exposure images.

3. The method of claim 1 , wherein the at least two of the first group of images generating the combined image includes at least one short-exposure image and at least one long-exposure image.

4. The method of claim 1 , wherein the at least two of the first group of images generating the combined image are short-exposure images.

5. The method of claim 1 , wherein obtaining the plurality of images comprises obtaining a first short-exposure image followed immediately by a long-exposure image followed immediately by a second short-exposure image.

6. The method of claim 1 , wherein obtaining the plurality of images comprises obtaining a first long-exposure image followed immediately by a short-exposure image followed immediately by a second long-exposure image.

7. The method of claim 1 , wherein the predetermined object is identified through exemplar model matching of a series of predetermined shapes.

8. The method of claim 1 , wherein the predetermined object is at least one of a human, a human face, a pet, or another live being.

9. An electronic device, comprising:
an image capture system;
a display unit;
a memory coupled to the image capture system and the display unit;
one or more processors coupled to the image capture system, the display unit and the memory, the one or more processors configured to execute instructions stored in the memory to—
obtain a plurality of images from a burst capture, wherein the plurality of images includes at least one or more short-exposure images and at least one or more long-exposure images;
identify a first group of images from the plurality of images, wherein the first group of images contains fewer images than the plurality of images;
analyze the first group of images to identify a predetermined object, the predetermined object having a corresponding location;
weight regions of at least some of the first group of images based, at least in part, on the corresponding location of the predetermined object;
combine at least two of the first group of images based, at least in part, on the weighted regions to generate a combined image; and
store the combined image in the memory.

10. The electronic device of claim 9 , wherein the first group of images comprises one or more short-exposure images.

11. The electronic device of claim 9 , wherein the predetermined object is identified through exemplar model matching of a series of predetermined shapes.

12. The electronic device of claim 9 , wherein the at least two of the first group of images generating the combined image are short-exposure images.

13. The electronic device of claim 9 , wherein the instructions to cause one or more processors to obtain the plurality of images comprises instructions to cause the one or more processors to obtain a first short-exposure image followed immediately by a long-exposure image followed immediately by a second short-exposure image.

14. The electronic device of claim 9 , wherein the instructions to cause the one or more processors to obtain the plurality of images comprise instructions to cause the one or more processors to obtain a first long-exposure image followed immediately by a short-exposure image followed immediately by a second long-exposure image.

15. A non-transitory program storage device comprising instructions stored thereon, the instructions readable by one or more processors and configured to cause one or more processors to:
obtain a plurality of images from a burst capture, wherein the plurality of images includes at least one or more short-exposure images and at least one or more long-exposure images;
identify a first group of images from the plurality of images, wherein the first group of images contains fewer images than the plurality of images;
analyze the first group of images to identify a predetermined object, the predetermined object having a corresponding location;
weight regions of at least some of the first group of images based, at least in part, on the corresponding location of the predetermined object;
combine at least two of the first group of images based, at least in part, on the weighted regions to generate a combined image; and
store the combined image in a memory.

16. The non-transitory program storage device of claim 15 , wherein the first group of images comprises one or more short-exposure images.

17. The non-transitory program storage device of claim 15 , wherein the predetermined object is identified through exemplar model matching of a series of predetermined shapes.

18. The non-transitory program storage device of claim 15 , wherein the at least two of the first group of images generating the combined image are short-exposure images.

19. The non-transitory program storage device of claim 15 , wherein the predetermined object is at least one of a human, a human face, a pet, or another live being.

Assignees

• Apple Inc

Inventors

• Molgaard Claus
• Tico Marius
• Toft Rolf
• Hubel Paul M

Classifications

• H04N23/6845 Primary

  by combination of a plurality of images sequentially taken · CPC title

• H04N23/743

  Bracketing, i.e. taking a series of images with varying exposure conditions · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N23/6811

  based on the image signal · CPC title