Image processing apparatus and image processing method which performs reflection removal processing by using polarization images

Patent metadata

Field	Value
Publication number	US-12088929-B2
Application number	US-202118002491-A
Country	US
Kind code	B2
Filing date	May 13, 2021
Priority date	Jun 29, 2020
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

A same region detector detects a same region of a processing target for each of a plurality of different viewpoints from polarization images in a plurality of polarization directions acquired for each of the viewpoints. The polarization images in the plurality of polarization directions acquired for each of the plurality of different viewpoints are, for example, polarization images acquired by imaging over a period of a plurality of frames in which a positional relationship between the processing target and a polarization image acquisition unit that acquires the polarization images changes. A polarization degree calculation unit calculates a polarization degree of the same region for each of the viewpoints on the basis of the polarization images in the plurality of polarization directions. A reflection removal unit performs reflection removal processing on the same region of the processing target by using the polarization images in the plurality of polarization directions of the viewpoint at which the polarization degree calculated by the polarization degree calculation unit is maximized. A reflection component can be removed even when an angle between a plane direction of a reflecting surface and an imaging direction is not clear.

First claim

The invention claimed is:

1. An image processing apparatus, comprising:
a same region detector configured to detect a same region of a processing target for each of a plurality of viewpoints from polarization images in a plurality of polarization directions for each of the plurality of viewpoints; and
an image processor configured to:
calculate a polarization degree of the same region for each of the plurality of viewpoints based on the polarization images in the plurality of polarization directions; and
perform reflection removal process on the same region of the processing target based on the polarization images in the plurality of polarization directions of a viewpoint of the plurality of viewpoints at which the calculated polarization degree is maximized.

2. The image processing apparatus according to claim 1 , wherein
the polarization images in the plurality of polarization directions for each of the plurality of viewpoints correspond to the polarization images based on imaging over a specific period of a plurality of frames, and
the specific period corresponds to a period of change in a positional relationship between the processing target and an image sensor that acquires the polarization images.

3. The image processing apparatus according to claim 2 , wherein the polarization images are the polarization images acquired by imaging the processing target that moves.

4. The image processing apparatus according to claim 2 , wherein the polarization images are the polarization images acquired by performing imaging by the image sensor that moves.

5. The image processing apparatus according to claim 1 , wherein the polarization images in the plurality of polarization directions for the plurality of viewpoints are the polarization images acquired by an image sensor, wherein the image sensor acquires the polarization images at a plurality of viewpoint positions and images the processing target.

6. The image processing apparatus according to claim 1 , wherein the image processor is further configured to calculate the polarization degree for each pixel unit.

7. The image processing apparatus according to claim 1 , wherein based on a position of the processing target being fixed, a position of the viewpoint at which the calculated polarization degree is maximized is stored with the position of the processing target.

8. The image processing apparatus according to claim 7 , wherein
the image processor is further configured to perform the reflection removal process based on the polarization images of the processing target, and
the polarization images is at the position of the viewpoint at which the calculated polarization degree is maximized.

9. An image processing method, comprising:
detecting, by a same region detector, a same region of a processing target for each of a plurality of viewpoints from polarization images in a plurality of polarization directions for each of the plurality of viewpoints;
calculating a polarization degree of the same region for each of the plurality of viewpoints based on the polarization images in the plurality of polarization directions; and
performing reflection removal process on the same region of the processing target based on the polarization images in the plurality of polarization directions of a viewpoint of the plurality of viewpoints at which the calculated polarization degree is maximized.

10. A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations, the operations comprising:
detecting a same region of a processing target for each of a plurality of viewpoints from polarization images in a plurality of polarization directions for each of the plurality of viewpoints;
calculating a polarization degree of the same region for each of the plurality of viewpoints based on the polarization images in the plurality of polarization directions; and
performing reflection removal processing on the same region of the processing target based on the polarization images in the plurality of polarization directions of a viewpoint of the plurality of viewpoints at which the calculated polarization degree is maximized.

Assignees

• Sony Group Corp

Inventors

• Hirasawa Yasutaka
• Kondo Yuhi
• Sun Legong
• Ono Taishi

Classifications

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• H04N23/76

  by influencing the image signals · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N23/81 Primary

  for suppressing or minimising disturbance in the image signal generation · CPC title