Dynamic bracketing operations for image stabilization

Patent metadata

Field	Value
Publication number	US-9357130-B2
Application number	US-201313909700-A
Country	US
Kind code	B2
Filing date	Jun 4, 2013
Priority date	Mar 15, 2013
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

Techniques are disclosed for selectively capturing, retaining, and combining multiple sub-exposure images or brackets to yield a final image having diminished motion-induced blur and good noise characteristics. More specifically, after or during the capture of N brackets, the M best may be identified for combining into a single output image, (N>M). As used here, the term “best” means those brackets that exhibit the least amount of relative motion with respect to one another—with one caveat: integer pixel shifts may be preferred over sub-pixel shifts.

First claim

The invention claimed is:

1. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors in an image capture device to:
receive an initial exposure time;
partition the initial exposure time into a plurality of sub-exposure times;
capture, for each sub-exposure time, a sub-exposure image and, at the same time, motion metadata corresponding to each sub-exposure image;
select a subset of the plurality of sub-exposure images based, at least in part, on comparing the sub-exposure images according to at least one motion distance criterion and the motion metadata associated with each sub-exposure image;
combine the subset of the sub-exposure images into a single image; and
store the single image in a memory.

2. The non-transitory program storage device of claim 1 , wherein the instructions to cause the one or more processors to receive an initial exposure time further comprise instructions to cause the one or more processors to determine whether image stabilization is beneficial is based, at least in part, on camera motion information and an image stabilization threshold.

3. The non-transitory program storage device of claim 1 , wherein the motion metadata comprises motion information from one or more motion sensors affixed to, part of the camera, or both.

4. The non-transitory program storage device of claim 1 , wherein the instructions to cause the one or more processors to select the subset of the plurality of sub-exposure images comprise instructions to cause the one or more processors to, for each of the plurality of sub-exposure images:
identify a motion metric value for a sub-exposure image;
select the sub-exposure image if the motion metric value is less than a threshold value determined from the at least one motion distance criterion; and
discard the sub-exposure image if the motion metric value is greater than, or equal to, the threshold value determined from the at least one motion distance criterion.

5. The non-transitory program storage device of claim 1 , wherein the instructions to cause the one or more processors to combine the subset of sub-exposure images comprise instructions to cause the one or more processors to:
average the sub-exposure images within the subset of sub-exposure images that are wholly coincident; and
perform a pixel-level analysis to identify objects having apparent motion between consecutively captured sub-exposure images within the subset of sub-exposure images for sub-exposure images that are not wholly coincident.

6. The non-transitory program storage device of claim 1 , wherein the at least one motion distance criterion represents a least amount of relative motion amongst the sub-exposure images for a N number of sub-exposure images.

7. The non-transitory program storage device of claim 6 , wherein the at least one motion distance criterion accounts for integer pixel shifts and sub-pixel shifts in representing the least amount of relative motion amongst the sub-exposure images.

8. The non-transitory program storage device of claim 1 , wherein the at least one motion distance criterion represents a histogram mode.

9. The non-transitory program storage device of claim 8 , wherein the instructions to cause the one or more processors to select the subset of the plurality of sub-exposure images comprise instructions to cause the one or more processors to:
generate a distance histogram for each sub-exposure image based at least on the motion metadata; and
compare the distance histogram for each sub-exposure image to the histogram mode.

10. An image capture method for a camera, comprising:
receiving an initial exposure time;
partitioning the initial exposure time into a plurality of sub-exposure times;
capturing, for each sub-exposure time, a sub-exposure image and, at the same time, motion metadata corresponding to each sub-exposure image;
selecting a subset of the plurality of sub-exposure images based, at least in part, on comparing the sub-exposure images according to at least one motion distance criterion and the motion metadata associated with each sub-exposure image;
combining the subset of the sub-exposure images into a single image; and
storing the single image in a memory.

11. The method of claim 10 , wherein the motion metadata comprises motion information from one or more motion sensors affixed to, part of the camera, or both.

12. The method of claim 10 , wherein the act of selecting a subset of the plurality of sub-exposure images comprises, for each of the plurality of sub-exposure images:
identifying a motion metric value for a sub-exposure image;
selecting the sub-exposure image if the motion metric value is less than a first threshold value determined from the at least one motion distance criterion; and
discarding the sub-exposure image if the motion metric value is greater than or equal to the threshold value determined from the at least one motion distance criterion.

13. The method of claim 10 , wherein the at least one motion distance criterion represents a least amount of relative motion amongst the sub-exposure images for a N number of sub-exposure images.

14. The method of claim 13 , wherein the at least one motion distance criterion accounts for integer pixel shifts and sub-pixel shifts in representing the least amount of relative motion amongst the sub-exposure images.

15. The method of claim 10 , wherein the at least one motion distance criterion represents a histogram mode, and wherein the method further comprises:
generating a distance histogram for each sub-exposure image based at least on the motion metadata; and
comparing the distance histogram for each sub-exposure image to the histogram mode.

16. A camera, comprising:
an image sensor;
one or more motion sensors affixed to, or a part of, the camera;
a memory operatively coupled to the image sensor; and
one or more processing elements operatively coupled to the image sensor, one or more motion sensors, memory and display, and configured to execute program instructions stored in the memory to:
receive an initial exposure time,
partition the initial exposure time into a plurality of sub-exposure times based, at least in part, on motion information from the one or more motion sensors,
capture, for each of the plurality of sub-exposure times, a sub-exposure image and at the same time motion metadata for each of the sub-exposure image from the one or more motion sensors,
select a subset of the plurality of sub-exposure images based, at least in part, on comparing the sub-exposure images according to at least one motion distance criterion and the motion metadata associated with each sub-exposure image,
combine the subset of the sub-exposure images into a single image, and
store the single image in the memory.

17. The camera of claim 16 , wherein the instructions to cause the one or more processors to select the subset of the plurality of sub-exposure images comprise instructions to cause the one or more processors to, for each of the plurality of sub-exposure images:
identify a motion metric value for a sub-exposure image;
select the sub-exposure image if the motion metric value is less than or equal to a threshold value determined from the at least one motion distance criterion; and
discard the sub-exposure image if the motion metric value is greater than the threshold value determined from the at least one motion distance criterion.

18. The camera of claim 16 , wherein the instructions to cause the one or more processing elements to combine the subse

Assignees

• Apple Inc

Inventors

• Sachs Todd
• Tico Marius
• Toft Rolf
• Hubel Paul
• Williams George

Classifications

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/743

  Bracketing, i.e. taking a series of images with varying exposure conditions · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N5/2356

  Electricity · mapped topic