Apparatus, method and computer program for image stabilization

Patent metadata

Field	Value
Publication number	US-9900512-B2
Application number	US-201314778330-A
Country	US
Kind code	B2
Filing date	Mar 18, 2013
Priority date	Mar 18, 2013
Publication date	Feb 20, 2018
Grant date	Feb 20, 2018

Abstract

An apparatus including an image sensor configured to capture an image and optics, wherein the image sensor and optics align along an optical axis; optical image stabilization circuitry configured to determine and apply a target optical adjustment to compensate during capture of an image, for movement of the optical axis as a consequence of detected movement of the apparatus; post-capture image stabilization circuitry configured to process the captured image using a residual optical adjustment to obtain a compensated captured image, wherein the residual optical adjustment depends upon a difference between a measured optical adjustment, achieved during capture of the image, and the determined target optical adjustment.

First claim

We claim:

1. An apparatus comprising:
an image sensor configured to capture an image and optics, wherein the image sensor and optics align along an optical axis;
optical image stabilization circuitry configured to determine and apply a target optical adjustment to compensate during capture of an image, for movement of the optical axis as a consequence of detected movement of the apparatus, wherein the target optical adjustment is a physical adjustment along the optical axis; and
post-capture image stabilization circuitry configured to process the captured image using a residual optical adjustment to obtain a compensated captured image, wherein the residual optical adjustment depends upon a difference between a measured optical adjustment, achieved during capture of the image, and the determined target optical adjustment.

2. An apparatus as claimed in claim 1 , comprising at least one sensor configured to measure, during capture of the image, a physical optical adjustment as a consequence of applying the target optical adjustment.

3. An apparatus as claimed in claim 1 , wherein the residual optical adjustment comprises both direction and magnitude.

4. An apparatus as claimed in claim 1 comprising digital image stabilization circuitry configured to image process the captured image using the residual optical adjustment to obtain a compensated captured image.

5. An apparatus as claimed in claim 1 , wherein the optical image stabilization circuitry is configured to apply the target optical adjustment by moving the optical axis relative to a scene to be captured during capture of the image.

6. An apparatus as claimed in claim 5 , wherein the optical image stabilization circuitry comprises at least one actuator configured to apply the target optical adjustment by physically moving the optical axis relative to a scene to be captured during capture of the image.

7. An apparatus as claimed in claim 1 , comprising a movement detector device configured to detect movement of the apparatus.

8. An apparatus as claimed in claim 7 , wherein the optical image stabilization circuitry is configured to determine a target optical adjustment using the output from the movement detector device, wherein the target optical adjustment compensates for the detected movement of the apparatus.

9. An apparatus as claimed in claim 1 , comprising residual optical adjustment circuitry configured to determine and store the residual optical adjustment, wherein the residual optical adjustment depends upon a difference between a measured optical adjustment, achieved during capture of the image, and the determined target optical adjustment.

10. An apparatus as claimed in claim 9 , wherein the residual optical adjustment circuitry configured to determine and store the residual optical adjustment, is configured to store the residual optical adjustment in a manner that records the temporal evolution of the residual optical adjustment signal relative to the timing of an exposure period for capturing the image.

11. An apparatus as claimed in claim 9 , wherein the image sensor provides at least one exposure period timing signal to the residual optical adjustment circuitry.

12. An apparatus as claimed in claim 11 , wherein the exposure period is determined before image capture starts.

13. An apparatus as claimed in claim 9 , wherein the residual optical adjustment circuitry configured to determine and store the residual optical adjustment, is configured to determine and store the residual optical adjustment over a continuous period corresponding to at least a portion of the exposure period for capturing the image.

14. An apparatus as claimed in claim 13 , wherein the residual optical adjustment circuitry configured to determine and store the residual optical adjustment, is configured to determine and store the residual optical adjustment over a continuous period corresponding to the exposure period for capturing the image.

15. An apparatus as claimed in claim 13 , wherein the residual optical adjustment circuitry configured to determine and store the residual optical adjustment, is configured to determine the residual optical adjustment during the exposure period for capturing the image and is configured to store the residual optical adjustment over a continuous period, during the exposure period for capturing the image, when the residual optical adjustment exceeds a threshold value.

16. A method comprising:
performing optical image stabilization by:
detecting movement of apparatus
determining target optical adjustment to compensate during capture of an image, for relative movement of the optical axis as a consequence of the detected movement of the apparatus, wherein the target optical adjustment is a physical adjustment along the optical axis;
using the target optical adjustment during capture of the image and then performing post-capture image stabilization by:
measuring an obtained optical adjustment that at least partially compensates during capture of an image for relative movement of the optical axis as a consequence of the movement of the apparatus;
using the target optical adjustment and the obtained optical adjustment to determine a residual optical adjustment indicative of a difference between the target optical adjustment and the obtained optical adjustment; and
processing the captured image using the residual optical adjustment to obtain a compensated captured image.

17. A method as claimed in claim 16 further comprising performing any of the functions enabled by the apparatus which comprises
an image sensor configured to
capture an image and optics, wherein the image sensor and optics align along an optical axis;
optical image stabilization circuitry configured to
determine and apply a target optical adjustment to compensate during capture of an image, for movement of the optical axis as a consequence of detected movement of the apparatus;
post-capture image stabilization circuitry configured to process the captured image using a residual optical adjustment to obtain a compensated captured image, wherein the residual optical adjustment depends upon a difference between a measured optical adjustment, achieved during capture of the image, and the determined target optical adjustment.

18. An apparatus comprising:
optical image stabilization circuitry configured to determine and apply a target optical adjustment to compensate during capture of an image of a scene, for relative movement of an image sensor as a consequence of detected movement, wherein the target optical adjustment is a physical adjustment along the optical axis; and
residual optical adjustment circuitry configured to determine and store a residual optical adjustment for obtaining a compensated captured image via processing, wherein the residual optical adjustment depends upon a difference between a measured optical adjustment, achieved during capture of the image, and the determined target optical adjustment.

19. A method comprising:
determining and applying a target optical adjustment to compensate during capture of an image of a scene, for relative movement of an image sensor as a consequence of detected movement, wherein the target optical adjustment is a physical adjustment along the optical axis; and
determining and storing a residual optical adjustment for obtaining a compensated captured image via post-capture processing, wherein the residual optical adjustment depends upon a difference between a measured optical adjustment, achieved during capture of the image, and the determined target o

Assignees

• Nokia Technologies Oy

Inventors

• Tuulos Eero
• Gren Juuso
• Muukki Mikko

Classifications

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/687 Primary

  by shifting the lens or sensor position · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• G03B5/00 Primary

  Adjustment of optical system relative to image or object surface other than for focusing · CPC title

• H04N5/23267

  Electricity · mapped topic