Image stabilization for electronic devices such as cameras

Patent metadata

Field	Value
Publication number	US-11218636-B2
Application number	US-202017068138-A
Country	US
Kind code	B2
Filing date	Oct 12, 2020
Priority date	Oct 10, 2018
Publication date	Jan 4, 2022
Grant date	Jan 4, 2022

Abstract

An electronic device includes an image sensor configured to receive light that is incident on the electronic device through a lens and generate image data, a gyroscope configured to generate rotation motion data of the electronic device, and an image stabilizer configured to generate translation motion data of the electronic device based on the image data received from the image sensor and to generate a stabilized image by compensating for a motion of the electronic device while capturing the image, such as vibration of a handheld device. The stabilized image is generated based on filtered translation motion data obtained by performing a first filtering operation on the translation motion data and filtered rotation motion data obtained by performing a second filtering operation on the rotation motion data received from the gyroscope.

First claim

What is claimed is:

1. An image signal processor comprising:
processing circuitry configured to,
generate translation motion data of an electronic device based on an image data received from an image sensor,
receive a rotation motion data from a gyroscope,
generate filtered translation motion data by performing a first filtering operation on the translation motion data,
generate filtered rotation motion data by performing a second filtering operation on the rotation motion data, and
generate a stabilized image based at least on the filtered translation motion data and the filtered rotation motion data, wherein the stabilized image is configured to compensate for motion of the electronic device while capturing an image.

2. The image signal processor of claim 1 , wherein the processing circuitry is configured to:
accumulate the translation motion data to generate accumulated translation motion data having a one-dimensional matrix form; and
generate the filtered translation motion data by performing the first filtering operation on the accumulated translation motion data.

3. The image signal processor of claim 1 , wherein the processing circuitry is configured to:
accumulate the translation motion data to generate accumulated translation motion data having a one-dimensional matrix form; and
generate the filtered translation motion data by dividing the accumulated translation motion data by a focal length of a lens and performing the first filtering operation.

4. The image signal processor of claim 1 , wherein the processing circuitry is configured to:
accumulate the rotation motion data to generate accumulated rotation motion data having a one-dimensional matrix form; and
generate the filtered rotation motion data by performing the second filtering operation on the accumulated rotation motion data.

5. The image signal processor of claim 4 , wherein,
the first filtering operation is a first low pass filtering operation, and
the processing circuitry is configured to,
generate low pass rotation motion data by performing a second low pass filtering operation on the accumulated rotation motion data, and
generate the filtered rotation motion data by subtracting the low pass rotation motion data from the accumulated rotation motion data.

6. The image signal processor of claim 1 , wherein the first filtering operation and the second filtering operation are complementary filtering operations.

7. The image signal processor of claim 1 , wherein,
the first filtering operation is a low pass filtering operation, and
the second filtering operation is a high pass filtering operation.

8. The image signal processor of claim 1 , wherein the processing circuitry is configured to accumulate the translation motion data.

9. The image signal processor of claim 1 , wherein,
the translation motion data comprises first axis-direction translation motion data and second axis-direction translation motion data, and
the rotation motion data comprises first axis-direction rotation motion data, second axis-direction rotation motion data, and third axis-direction rotation motion data.

10. The image signal processor of claim 9 , wherein,
the filtered translation motion data comprises filtered first axis-direction translation motion data and filtered second axis-direction translation motion data,
the filtered rotation motion data comprises filtered first axis-direction rotation motion data, filtered second axis-direction rotation motion data, and filtered third axis-direction rotation motion data, and
the processing circuitry is configured to,
generate first motion information by combining the filtered first axis-direction translation motion data and the filtered first axis-direction rotation motion data,
generate second motion information by combining the filtered second axis-direction translation motion data and the filtered second axis-direction rotation motion data, and
generate the filtered third axis-direction rotation motion data as third motion information.

11. The image signal processor of claim 1 , wherein the processing circuitry is configured to:
compare a sensing frequency of the gyroscope with an imaging frequency at which the image sensor provides the image data;
down-sample the translation motion data in response to the imaging frequency being higher than the sensing frequency; and
down-sample the rotation motion data in response to the sensing frequency being higher than the imaging frequency.

12. The image signal processor of claim 1 , wherein the processing circuitry is configured to activate an operation of the image signal processor in response to a camera starting one of a moving image photographing operation, a time lapse photographing operation, or a panorama photographing operation each using a lens.

13. An image signal processor comprising:
processing circuitry configured to,
generate translation motion data of an electronic device based on an image data received from an image sensor,
generate a motion information based on the translation motion data and rotation motion data received from a gyroscope, and
compensate for a motion of the electronic device with respect to the image data to generate compensated image data, based on the motion information.

14. The image signal processor of claim 13 , wherein the processing circuitry is configured to generate the motion information based at least on filtered translation motion data and filtered rotation motion data.

15. The image signal processor of claim 14 , wherein the processing circuitry is configured to:
accumulate the translation motion data;
generate the filtered translation motion data by performing a first filtering operation on the accumulated translation motion data; and
generate the filtered rotation motion data by performing a second filtering operation on the rotation motion data received from the gyroscope.

16. The image signal processor of claim 15 , wherein,
the first filtering operation is a low pass filtering operation, and
the second filtering operation is a high pass filtering operation.

17. An image stabilizing method comprising:
generating translation motion data of an electronic device based on an image data received from an image sensor;
receiving rotation motion data from a gyroscope;
generating a motion information based on the translation motion data and the rotation motion data; and
generating compensated image data by compensating for a motion of the electronic device with respect to the image data based on the motion information.

18. The image stabilizing method of claim 17 further comprising:
generating accumulated translation motion data by accumulating the translation motion data based on the image data;
generating filtered translation motion data by filtering the accumulated translation motion data using a first filter;
generating accumulated rotation motion data by accumulating the rotation motion data obtained by the gyroscope to; and
generating filtered rotation motion data by filtering the accumulated rotation motion data using a second filter.

19. The image stabilizing method of claim 18 further comprising:
generating a stabilized image based at least on the filtered translation motion data and the filtered rotation motion data, wherein the stabilized image is configured to compensate for motion of the electronic device while capturing an image.

20. The image stabilizing method of claim 19 , wherein,
each of the accumulated translation motion data and the accum

Assignees

• Samsung Electronics Co Ltd

Inventors

• Guo Kai
• Choi Seung-Won

Classifications

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title