Camera device and electronic device including the same

Patent metadata

Field	Value
Publication number	US-11941848-B2
Application number	US-202017421927-A
Country	US
Kind code	B2
Filing date	Jan 10, 2020
Priority date	Jan 11, 2019
Publication date	Mar 26, 2024
Grant date	Mar 26, 2024

Abstract

The present disclosure relates to a camera device. The camera device and an electronic device including the same according to an embodiment of the present disclosure include: a color camera; an IR camera; and a processor configured to extract a first region of a color image from the color camera, to extract a second region of an IR image from the IR camera, to calculate error information based on a difference between a gradient of the first region and a gradient of the second region, to compensate for at least one of the color image and the IR image based on the calculated error information, and to output a compensated color image or a compensated IR image.

First claim

What is claimed is:

1. A camera device comprising:
a color camera;
an infrared (IR) camera;
a flexible connection member connecting the color camera and the IR camera;
a sensor device configured to sense a movement or impact: and
a processor configured to:
in response to the movement or the impact sensed by the sensor device being less than a first level, calculate three-dimensional (3D) error information based on a color image from the color camera and an IR image from the IR camera, compensate for at least one of the color image and the IR image based on the calculated 3D error information, and output a compensated color image or a compensated IR image, and
in response to the movement or the impact sensed by the sensor device being greater than or equal to the first level, extract a first region of a color image from the color camera, extract a second region of an IR image from the IR camera, calculate error information based on a difference between a gradient of the first region and a gradient of the second region, compensate for at least one of the color image and the IR image based on the calculated error information, and output a compensated color image or a compensated IR image.

2. The camera device of claim 1 , wherein the processor is configured to:
in response to the movement or the impact sensed by the sensor device being greater than or equal to the first level, normalize the gradient of the first region and the gradient of the second region, and compensate the 3D error information based on a difference between the normalized gradient of the first region and the normalized gradient of the second region, and
based on the compensated 3D error information, compensate for at least one of the color image and the IR image, and output the compensated color image or the compensated IR image.

3. The camera device of claim 1 , wherein in response to the movement or the impact sensed by the sensor device being less than the first level, the processor is configured to update the calculated 3D error information based on pre-stored 3D error information and the calculated 3D error information, compensate for at least one of the color image and the IR image based on the updated 3D error information, and output the compensated color image or the compensated IR image.

4. The camera device of claim 1 , wherein the processor is configured to:
in response to the movement or the impact sensed by the sensor device being less than the first level, upscale the IR image,
extract the first region of the color image from the color camera and the second region from the upscaled IR image,
calculate the 3D error information based on the difference between the gradient of the first region and the gradient of the second region, and
based on the calculated 3D error information, compensate for at least one of the color image and the IR image, and output the compensated color image or the compensated IR image.

5. The camera device of claim 1 , wherein the first region and the second region comprise a face region.

6. The camera device of claim 1 , wherein the first gradient or the second gradient respectively comprises gradients of rotation information, translation information, or shift information of the color image or the IR image.

7. The camera device of claim 1 , wherein in response to the movement or the impact sensed by the sensor device being greater than or equal to the first level, the processor is configured to extract the first region of the color image from the color camera and the second region of the IR image from the IR camera, compensate the 3D error information based on a difference between a gradient of a luminance component of the first region and a gradient of a luminance component of the second region, compensate for at least one of the color image and the IR image based on the compensated 3D error information, and output the compensated color image or the compensated IR image.

8. The camera device of claim 1 , wherein the processor comprises:
a feature point analyzer configured to analyze each feature point of the color image from the color camera and the IR image from the IR camera;
an external variable calculator configured to calculate an external variable based on each feature point information analyzed by the feature point analyzer;
an error calculator configured to calculate 3D error information based on the calculated external variable; and
an error corrector configured to correct an error based on the 3D error information.

9. The camera device of claim 8 , wherein, based on the each feature point information analyzed by the feature point analyzer, the external variable calculator is configured to calculate rotation information, translation information, or shift information of each of the color image and the IR image.

10. The camera device of claim 9 , wherein based on the rotation information, the translation information, or shift information of each of the color image and the IR image, the error calculator is configured to calculate relative rotation information, relative translation information, or relative shift information between the color image and the IR image.

11. An electronic device comprising a camera device,
wherein the camera device comprises:
a color camera;
an infrared (IR) camera;
a flexible connection member connecting the color camera and the IR camera;
a sensor device configured to sense a movement or impact: and
a processor configured to:
in response to the movement or the impact sensed by the sensor device being less than a first level, calculate three-dimensional (3D) error information based on a color image from the color camera and an IR image from the IR camera, compensate for at least one of the color image and the IR image based on the calculated 3D error information, and output a compensated color image or a compensated IR image, and
in response to the movement or the impact sensed by the sensor device being greater than or equal to the first level, extract a first region of a color image from the color camera, extract a second region of an IR image from the IR camera, calculate error information based on a difference between a gradient of the first region and a gradient of the second region, compensate for at least one of the color image and the IR image based on the calculated error information, and output a compensated color image or a compensated IR image.

12. The electronic device of claim 11 , wherein the processor is configured to:
in response to the movement or the impact sensed by the sensor device being greater than or equal to the first level, normalize the gradient of the first region and the gradient of the second region, and compensate the 3D error information based on a difference between the normalized gradient of the first region and the normalized gradient of the second region, and
based on the compensated 3D error information, compensate for at least one of the color image and the IR image, and output the compensated color image or the compensated IR image.

13. The electronic device of claim 11 , wherein in response to the movement or the impact sensed by the sensor device being less than the first level, the processor is configured to update the 3D error information based on pre-stored 3D error information and the calculated 3D error information, compensate for at least one of the color image and the IR image based on the updated 3D error information, and output the compensated color image or the compensated IR image.

14. The electronic device of claim 11 , wherein the processor is configured to:
in response to the movement or the impact sensed by the sens

Assignees

• Lg Electronics Inc

Inventors

• Kang Yunsuk
• Park Chanyong
• Lee Eunsung

Classifications

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/20

  for generating image signals from infrared radiation only · CPC title

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title