Apparatus for stabilizing digital image, operating method thereof, and electronic device having the same

Patent metadata

Field	Value
Publication number	US-11611708-B2
Application number	US-202117385601-A
Country	US
Kind code	B2
Filing date	Jul 26, 2021
Priority date	Feb 24, 2020
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

A method of operating a digital image stabilization apparatus including acquiring motion information by a first camera for capturing an image of a first field of view, controlling a prism of a second camera using the motion information, acquiring a main image and a peripheral image by a second camera for capturing an image of a second field of view, the second field of view being narrower than the first field of view, and generating a stabilized frame image related to the main image using the motion information and the peripheral image may be provided.

First claim

What is claimed is:

1. An electronic device comprising:
a first camera configured to capture a first image with a first field of view;
a second camera configured to capture a second image with a second field of view using a movable prism, the second field of view of the second camera being narrower than the first field of view of the first camera; and
a processor configured to control movement operation of the movable prism of the second camera based on motion information estimated from the first image from the first camera, to control the second camera to capture a peripheral image of the second image with the movable prism moved, and to perform digital image stabilization on the second image from the second camera using the peripheral image from the second camera.

2. The electronic device of claim 1 , wherein
the electronic device further comprises a third camera with a third field of view, the third field of view being narrower than the second field of view, and
the first camera, the second camera and the third camera are disposed in a rear surface of the electronic device.

3. The electronic device of claim 1 , wherein the second camera is a main camera for capturing video.

4. The electronic device of claim 1 , wherein the processor is further configured to perform the digital image stabilization, to attenuate or remove vibrations from the second image, using the peripheral image as a margin region of a video stabilization algorithm.

5. The electronic device of claim 3 , wherein a resolution of the first camera is lower than that of the second camera.

6. The electronic device of claim 4 , wherein the first camera includes a Dynamic Vision Sensor (DVS) and the second camera module-includes an RGB sensor.

7. The electronic device of claim 1 , wherein
the first camera includes a first optical lens assembly arranged in a first direction parallel to a thickness direction of the electronic device, and
the second camera includes a second optical lens assembly arranged in a second direction perpendicular to the first direction.

8. The electronic device of claim 7 , wherein the movable prism of the second camera is configured to convert an optical axis direction from the first direction to the second direction and move at least one of upward, downward, leftward, or rightward direction based on the estimated motion information.

9. The electronic device of claim 8 , wherein a number of one or more peripheral images including the peripheral image is determined according to an illumination value such that,
when the illumination value is greater than a reference value, the number of the one or more peripheral images is greater than two, and
when the illumination value is less than the reference value, the number of the one or more peripheral images is one.

10. The electronic device of claim 9 , wherein
the estimated motion information includes a movement direction of an object included in the first image, and
the peripheral images of the second image are acquired by moving the movable prism based on the estimated motion information.

11. The electronic device of claim 10 , wherein the stabilized second image includes an entirety of the second image within the second field of view.

12. An electronic device including at least one processor that is configured to perform operations on a first image of a first field of view transferred from a first camera and a second image of a second field of view transferred from a second camera, the operations comprising:
acquiring motion information from the first image, the second field of view of the second camera being narrower than the first field of view of the first camera;
controlling movement of a prism of the second camera to capture a peripheral image of the second image based on the acquired motion information; and
stabilizing the second image from the second camera using the peripheral image from the second camera,
wherein the stabilizing does not cause any image loss of the second image within the second field of view,
the first camera includes a first optical lens assembly arranged in a first direction for the first field of view, and
the second camera includes a second optical lens assembly arranged in a second direction perpendicular to the first direction for the second field of view.

13. The electronic device of claim 12 , wherein
the first camera includes a first optical lens assembly arranged in a first direction for the first field of view, and
the second camera includes a second optical lens assembly arranged in a second direction perpendicular to the first direction for the second field of view.

14. The electronic device of claim 12 , wherein
the motion information includes a shaking direction of the first camera, and
the controlling includes moving the prism to capture the peripheral image based on the shaking direction.

15. The electronic device of claim 12 , wherein
the operations further comprises receiving acceleration information from a gyro sensor, and
the acquired motion information is related to a movement of an object included in the first image.

16. The electronic device of claim 15 , wherein the second image is captured when the prism of the second camera is located in a reference position.

17. The electronic device of claim 16 , wherein the peripheral image is captured when the prism is moved from the reference position to one of upward, downward, leftward, or rightward direction.

18. The electronic device of claim 16 , wherein the stabilizing the second image does not include up-scaling operation deteriorating image quality of the second image.

19. The electronic device of claim 12 , wherein the stabilizing the second image does not cause any loss of the second image within the second field of view.

20. A camera module comprising:
a first camera configured to capture a first image with a first field of view in a first direction;
a second camera configured to capture a second image with a second field of view using a movable prism of the second camera, the second field of view of the second camera being narrower than the first field of view of the first camera; and
a first processor configured to
control a second processor to move the movable prism of the second camera and cause the second camera to capture a peripheral image of the second image with the movable prism moved, and
perform digital image stabilization on the second image from the second camera using the peripheral image from the second camera,

wherein the second camera includes,
the movable prism configured to convert an optical direction of the second image paralleling to the first direction to a second direction,
an image sensor configured to acquire the second image in the second direction, and
an optical lens assembly configured to include a zoom magnification of a section according to the second field of view, and
the second processor configured to control a driving direction of the movable prism based on motion information estimated from the first image from the first camera under a control of the first processor.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yi Eunjik
• JEONG Sooman
• Kim Heeshin
• Sung Hwansun
• Lee Suyoung
• Lee Jeongwook

Classifications

• H04N5/76

  Television signal recording · CPC title

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• H04N5/77 Primary

  between a recording apparatus and a television camera · CPC title

• H04N23/6845 Primary

  by combination of a plurality of images sequentially taken · CPC title

• H04N23/687

  by shifting the lens or sensor position · CPC title