Statistics-based electronic image stabilization

Patent metadata

Field	Value
Publication number	US-11363201-B2
Application number	US-202117237246-A
Country	US
Kind code	B2
Filing date	Apr 22, 2021
Priority date	Apr 28, 2020
Publication date	Jun 14, 2022
Grant date	Jun 14, 2022

Abstract

Methods and apparatus, including computer program products, for selecting a cropping area in a sequence of images captured by a camera. First external motion data including data of external factors affecting movement of the camera is correlated with internal motion data describing a movement of the camera. The first external motion data and the internal motion data are obtained during a first time interval. Second external motion data is obtained during a second time interval subsequent to the first time interval. A sequence of images being captured during a third time interval is stabilized by selecting a cropping area based on the correlation and the second external motion data. The cropping area remains constant and maximized in size with respect to the field of view of the images during the third time interval. The third time interval is subsequent to, or partly overlapping with, the second time interval.

First claim

The invention claimed is:

1. A method for selecting a cropping area in a sequence of images captured by a camera, comprising:
correlating first external motion data with internal motion data, wherein the first external motion data and the internal motion data are obtained during a first time interval, wherein the first external motion data comprises data of external factors affecting movement of the camera and wherein the internal motion data comprises data describing a movement of the camera;
obtaining second external motion data during a second time interval subsequent to the first time interval; and
stabilizing a sequence of images being captured during a third time interval, the third time interval being either subsequent to the second time interval or partly overlapping with the second time interval, by selecting a cropping area based on the correlation and the second external motion data, wherein the cropping area remains constant and maximized in size with respect to the field of view of the images in the sequence of images captured by the camera during the third time interval.

2. The method of claim 1 , wherein the first and second time intervals, respectively, are approximately 15 minutes or longer.

3. The method of claim 1 , wherein the correlating of the first external motion data with the internal motion data comprises:
for each point in time when a sensor readout occurs in the first time interval, correlating a respective first external motion datum with a corresponding internal motion datum.

4. The method of claim 1 , wherein obtaining the second external motion data comprises:
obtaining the second external motion data as a prediction based on the first external motion data and a prediction model.

5. The method of claim 1 , wherein:
the first and second external motion data includes one or more of: wind data and ground vibration data, and
the internal motion data includes one or more of: gyro data, accelerometer data, and motion estimation by image processing.

6. The method of claim 5 , wherein the motion estimation by image processing is done using a digital image stabilization algorithm.

7. The method of claim 1 , wherein the obtaining of the first and second external motion data, respectively, comprises:
receiving the first and second external motion data, respectively, from one or more of: a motion sensor and an external weather service.

8. The method of claim 7 , wherein the motion sensor includes one of: a wind sensor, a ground vibration sensor, and a second camera.

9. The method of claim 1 , further comprising:
providing a recommendation, based on the second external motion data and the correlation, about whether image stabilization should be based on one or more of: pitch, jaw, and roll.

10. The method of claim 1 , further comprising:
in response to detecting a change in second external motion data that exceeds a predetermined threshold, selecting a different cropping area based on the correlation and the changed second external motion data.

11. The method of claim 1 , wherein selecting a cropping area further considers a variable stabilizer margin that is dependent on the field of view of the images in the sequence of images.

12. A system for selecting a cropping area in a sequence of images captured by a camera, comprising:
a camera including one or more internal motion sensors;
one or more sensors configured to measure external factors affecting movement of the camera; and
a processing unit including a memory and a processor,
wherein the memory contains instructions that when executed by the processor causes the processor to perform a method that includes:
correlating first external motion data with internal motion data, wherein the first external motion data and the internal motion data are obtained during a first time interval, wherein the first external motion data comprises data of external factors affecting movement of the camera and wherein the internal motion data comprises data describing a movement of the camera;
obtaining second external motion data during a second time interval subsequent to the first time interval; and
stabilizing a sequence of images being captured during a third time interval, the third time interval being either subsequent to the second time interval or partly overlapping with the second time interval, by selecting a cropping area based on the correlation and the second external motion data, wherein the cropping area remains constant and maximized in size with respect to the field of view of the images in the sequence of images captured by the camera during the third time interval.

13. A computer program product for selecting a cropping area in a sequence of images captured by a camera, comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal, the program instructions being executable by a processor to perform a method comprising:
correlating first external motion data with internal motion data, wherein the first external motion data and the internal motion data are obtained during a first time interval, wherein the first external motion data comprises data of external factors affecting movement of the camera and wherein the internal motion data comprises data describing a movement of the camera;
obtaining second external motion data during a second time interval subsequent to the first time interval; and
stabilizing a sequence of images being captured during a third time interval, the third time interval being either subsequent to the second time interval or partly overlapping with the second time interval, by selecting a cropping area based on the correlation and the second external motion data, wherein the cropping area remains constant and maximized in size with respect to the field of view of the images in the sequence of images captured by the camera during the third time interval.

Assignees

• Axis Ab

Inventors

• Nyström Johan
• Yuan Song

Classifications

• H04N23/682 Primary

  Vibration or motion blur correction · CPC title

• H04N23/684

  performed by controlling the image sensor readout, e.g. by controlling the integration time · CPC title

• H04N23/681 Primary

  Motion detection · CPC title

• H04N23/6842

  by controlling the scanning position, e.g. windowing · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title