Non-same camera based image processing apparatus

Patent metadata

Field	Value
Publication number	US-11589028-B2
Application number	US-201917418185-A
Country	US
Kind code	B2
Filing date	Aug 23, 2019
Priority date	Feb 1, 2019
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

The present invention provides an image processing apparatus comprising: a first camera obtaining a true-color image by capturing a subject; a second camera spaced apart from the first camera and obtaining an infrared image by capturing the subject; and a control unit connected to the first camera and the second camera, wherein the control unit matches the true-color image and the infrared image and obtains three-dimensional information of the subject by using the matched infrared image in a region corresponding to the matched true-color image and a valid pixel.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
cameras configured to detect different wavelength bands, wherein: a first camera among the cameras is configured to obtain a true color image by capturing a subject; a second camera among the cameras is configured to obtain an infrared image by capturing the subject, wherein the second camera is spaced apart from the first camera; and
a controller configured to:
match the obtained true color image and the obtained infrared image;
obtain three-dimensional information about the captured subject in an area corresponding to effective pixels of the matched true color image from the matched infrared image; obtain two-dimensional information about an area corresponding to abnormal pixels of the matched true color image from the matched infrared image,
wherein the obtained two-dimensional information comprises information about a presence of a specific object in the area corresponding to the abnormal pixels;
convert the abnormal pixels into the effective pixels based on raw data for obtaining the true color image based on a detection of the presence of the specific object from the two-dimensional information; and
obtain three-dimensional information about the specific object present in an area corresponding to the converted effective pixels.

2. The image processing apparatus of claim 1 , wherein the effective pixels correspond to pixels that absorb light greater than or equal to a minimum amount of light for identifying an object and less than or equal to a saturation amount of light.

3. The image processing apparatus of claim 1 , wherein the controller is further configured to coordinate a representative pixel of the captured subject among the effective pixels to the obtained infrared image, wherein the three-dimensional information is obtained based on a stereo vision operation.

4. The image processing apparatus of claim 3 , wherein the representative pixel comprises at least one of an outline pixel of the captured subject or a center pixel of the captured subject.

5. The image processing apparatus of claim 1 , wherein the controller is further configured to determine whether a shape of a specific object corresponds to the specific object based on information about a related area in the matched infrared image and based on recognition of the shape of a specific object in the area corresponding to the effective pixels.

6. An image processing apparatus comprising:
cameras configured to detect different wavelength bands, wherein: a first camera among the cameras is configured to obtain a true color image by capturing a subject a second camera among the cameras is configured to obtain an infrared image by capturing the subject, wherein the second camera is spaced apart from the first camera; and
a controller configured to:
match the obtained true color image and the obtained infrared image;
obtain three-dimensional information about the captured subject in an area corresponding to effective pixels of the matched true color image from the matched infrared image;
obtain two-dimensional information about an area corresponding to abnormal pixels of the matched true color image from the matched infrared image;
wherein the obtained two-dimensional information comprises information about a presence of a specific object in the area corresponding to the abnormal pixels; and
convert the abnormal pixels into effective pixels by adjusting at least one of an aperture, an exposure time, or an ISO of the first camera based on a detection of the presence of the specific object from the two-dimensional information.

7. The image processing apparatus of claim 6 , wherein the controller is further configured to obtain three-dimensional information about the specific object present in the area corresponding to the abnormal pixels after adjusting the at least one of the aperture, the exposure time, or the ISO of the first camera.

8. The image processing apparatus of claim 1 , further comprising an output interface, wherein the controller is further configured to output information about the specific object to a user through the output interface based on a detection of the specific object from the two-dimensional information.

9. The image processing apparatus of claim 8 , wherein the output interface further comprises a display configured to display the obtained true color image, and wherein the controller is further configured to cause on the display a display of an infrared image of the specific object in the area corresponding to the abnormal pixels.

10. The image processing apparatus of claim 1 , wherein the first camera corresponds to a camera configured to detect visible light and near-infrared (NIR) light.

11. The image processing apparatus of claim 1 , wherein the second camera corresponds to a camera configured to detect at least one of far-infrared (FIR) light, mid-wave infrared (MWIR) light, or short-wavelength infrared (SWIR) light.

Assignees

• Lg Electronics Inc

Inventors

• Lee Chaedeok
• Park Jongmyoung
• Seo Sangwon
• Joo Woodeok

Classifications

• H04N23/20

  for generating image signals from infrared radiation only · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• G06V10/147

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• G06V10/143

  Sensing or illuminating at different wavelengths · CPC title