Image processing apparatus and ranging apparatus

Patent metadata

Field	Value
Publication number	US-11587261-B2
Application number	US-201815905618-A
Country	US
Kind code	B2
Filing date	Feb 26, 2018
Priority date	Sep 8, 2017
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

According to one embodiment, an image processing apparatus includes a buffer and processing circuitry. The buffer stores first and second images capturing an object. The circuitry calculates at least one of a first distance to the object in the first image and a second distance to the object in the second image by using a correction parameter for correcting at least one of influences caused by ambient light, a reflection characteristic of the object, or a color of the object, calculates three-dimensional coordinates of the object on a relative scale by using the first and second images, and calculates three-dimensional coordinates of the object on a real scale based on at least one of the first and second distances, and the three-dimensional coordinates of the object on the relative scale.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a buffer configured to store a first image capturing an object and a second image capturing the object, the first image being different from the second image, the first image and the second image being captured by an image capturing device; and
processing circuitry configured to:
calculate at least one of a first distance to the object in the first image and a second distance to the object in the second image, the first and second distances each including a correction parameter for correcting at least one of influences caused by ambient light, a reflection characteristic of the object, or a color of the object;
calculate three-dimensional coordinates of the object on a relative scale by using only the first image and the second image;
calculate the correction parameter and a scale parameter for converting the three-dimensional coordinates on the relative scale into three-dimensional coordinates on a real scale so that at least one of the first distance and the second distance, and the three-dimensional coordinates of the object on the relative scale indicate a same position in a real space; and
calculate the three-dimensional coordinates of the object on the real scale by using the calculated correction parameter and the calculated scale parameter.

2. The image processing apparatus of claim 1 , wherein the processing circuitry is configured to calculate the three-dimensional coordinates of the object on the real scale based on the calculated scale parameter and the three-dimensional coordinates on the relative scale.

3. The image processing apparatus of claim 1 ,
wherein the first image comprises a first wavelength component image and a second wavelength component image,
the second image comprises a third wavelength component image and a fourth wavelength component image,
the processing circuitry is further configured to:
calculate a first blur correction amount used for correcting a blur included in the first wavelength component image to a blur included in the second wavelength component image;
calculate a second blur correction amount used for correcting a blur included in the third wavelength component image to a blur included in the fourth wavelength component image;
calculate the first distance based on the first blur correction amount; and
calculate the second distance based on the second blur correction amount.

4. The image processing apparatus of claim 3 , wherein:
at least one of the blur included in the first wavelength component image and the blur included in the second wavelength component image has a point-asymmetric shape, and
at least one of the blur included in the third wavelength component image and the blur included in the fourth wavelength component image has a point-asymmetric shape.

5. The image processing apparatus of claim 3 , wherein the processing circuitry is configured to:
calculate the correction parameter and a scale parameter that is used for converting the three-dimensional coordinates on the relative scale into the three-dimensional coordinates on the real scale by using at least one of the first distance and the second distance, and the three-dimensional coordinates on the relative scale; and
calculate the three-dimensional coordinates of the object on the real scale based on the calculated correction parameter and the first blur correction amount or based on the calculated correction parameter and the second blur correction amount.

6. The image processing apparatus of claim 5 , wherein the processing circuitry is configured to calculate the correction parameter for each corresponding point between the first image and the second image.

7. The image processing apparatus of claim 5 , wherein the processing circuitry is configured to calculate the correction parameter for each area having a similar color on the first image or the second image.

8. The image processing apparatus of claim 5 , wherein the processing circuitry is configured to calculate the correction parameter common in pixels of the first image and pixels of the second image.

9. The image processing apparatus of claim 1 , wherein the processing circuitry is further configured to calculate one or more parameters indicative of a relative positional relationship between a position and posture at which the first image is captured and a position and posture at which the second image is captured.

10. The image processing apparatus of claim 1 , wherein distance information to the object at a time of capturing is encoded in each of the first image and the second image.

11. The image processing apparatus of claim 1 , wherein the first image and the second image are both obtained by an optical system.

12. A ranging apparatus comprising:
an image processing apparatus comprising:
a buffer configured to store a first image capturing an object and a second image capturing the object, the first image different from the second image; and
a processing circuitry configured to:
calculate at least one of a first distance to the object in the first image and a second distance to the object in the second image, the first and second distances each including a correction parameter for correcting at least one of influences caused by ambient light, a reflection characteristic of the object, or a color of the object;
calculate three-dimensional coordinates of the object on a relative scale by using only the first image and the second image;
calculate the correction parameter and a scale parameter for converting the three-dimensional coordinates on the relative scale into three-dimensional coordinates on a real scale so that at least one of the first distance and the second distance, and the three-dimensional coordinates of the object on the relative scale indicate a same position in a real space; and
calculate the three-dimensional coordinates of the object on the real scale by using the calculated correction parameter and the calculated scale parameter;
an image capturing device configured to capture the first image and the second image.

13. The ranging apparatus of claim 12 , wherein the image capturing device comprises:
a lens;
an image sensor comprising at least two types of imaging elements configured to receive light rays having different wavelength bands; and
a filter provided on an optical path of light rays penetrating the lens and entering the image sensor, comprising at least two filter areas which transmit light rays having different wavelength bands.

Assignees

• Toshiba Kk

Inventors

• Yamaguchi Jun
• Moriuchi Yusuke
• Mishima Nao

Classifications

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06T7/514

  from specularities · CPC title

• G06T7/55 Primary

  from multiple images · CPC title

• G06T7/97 Primary

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• G06T2200/04

  involving 3D image data · CPC title