Information processing device, imaging device, apparatus control system, movable body, information processing method, and computer program product

Patent metadata

Field	Value
Publication number	US-11030761-B2
Application number	US-201716347938-A
Country	US
Kind code	B2
Filing date	Nov 2, 2017
Priority date	Nov 30, 2016
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

An information processing device includes an image analyzing unit. The image analyzing unit includes an object region setting unit configured to set an object region, for image information obtained by capturing an imaging range, the object region corresponding to an object existing in the imaging range; a luminance information acquiring unit configured to acquire luminance information indicating luminance in the imaging range; and a correcting unit configured to correct the object region based on the luminance information in a luminance detection region set in a lower part of the object region.

First claim

The invention claimed is:

1. An information processing device, comprising:
processing circuitry configured to
set an initial object region within image information obtained by capturing an imaging range, the initial object region corresponding to an object existing in the imaging range;
acquire luminance information indicating luminance in the imaging range; and
adjust the initial object region to obtain an adjusted object region based on the acquired luminance information within a luminance detection region that is set in a lower part of the initial object region, wherein the processing circuitry is further configured to
acquire distance information indicating a distance between the object and an apparatus equipped with a stereo camera that acquires the image information and generates look-down view information based on the distance information, wherein the distance information is disparity information,
set the initial object region based on the look-down view information,
set the initial object region in a disparity image indicating a distribution of disparities in the imaging range based on the disparity information,
generate U map data indicating a U map, in which a combination of a first axial position, a second axial position, and a disparity in the disparity image is converted to two-dimensional histogram information, based on the disparity information,
generate real U map data indicating a real U map, in which a first axis in the U map is converted to an actual distance and the disparity on a second axis is converted to a decimated disparity value at a decimation rate corresponding to the actual distance, based on the U map data, and
detect an isolated region corresponding to the object in the real U map, and set the initial object region based on the isolated region.

2. The information processing device according to claim 1 , wherein the processing circuitry is further configured to set the initial object region based on the luminance information.

3. The information processing device according to claim 1 , wherein the processing circuitry is further configured to adjust the initial object region based on a region in which luminance indicated by the acquired luminance information is smaller than a predetermined value within the luminance detection region.

4. The information processing device according to claim 1 , wherein the processing circuitry is further configured to:
determine whether the object is a vehicle, and
when the object is determined to be the vehicle, adjust the initial object region based on the luminance information within the luminance detection region.

5. The information processing device according to claim 1 wherein the processing circuitry is further configured to:
detect a shape of a road surface based on the distance information, and
correct the shape of the road surface based on a lower end position of the adjusted object region.

6. An imaging device comprising the information processing device according to claim 1 .

7. An apparatus control system comprising:
the information processing device according to claim 1 ; and
an apparatus control circuit configured to control an operation of an apparatus based on information in the adjusted object region.

8. A movable body comprising:
the apparatus control system according to claim 7 , wherein the movable body is controlled by the apparatus control circuit.

9. The information processing device of claim 1 , wherein the processing circuitry is further configured to adjust a location of the bottom boundary of the initial object region based on a comparison between an average luminance of at least one row of pixels in the luminance detection region and an average luminance of all pixels within the luminance detection region.

10. An information processing method, comprising:
setting an initial object region within image information obtained by capturing an imaging range, the initial object region corresponding to an object existing in the imaging range;
acquiring luminance information indicating luminance in the imaging range; and
adjusting the initial object region to obtain an adjusted object region, based on the acquired luminance information in a luminance detection region set within a lower part of the initial object region, wherein the method further comprises
acquiring distance information indicating a distance between the object and an apparatus equipped with a stereo camera that acquires the image information and generates look-down view information based on the distance information, wherein the distance information is disparity information;
setting the initial object region based on the look-down view information;
setting the initial object region in a disparity image indicating a distribution of disparities in the imaging range based on the disparity information;
generating U map data indicating a U map, in which a combination of a first axial position, a second axial position, and a disparity in the disparity image is converted to two-dimensional histogram information, based on the disparity information;
generating real U map data indicating a real U map, in which a first axis in the U map is converted to an actual distance and the disparity on a second axis is converted to a decimated disparity value at a decimation rate corresponding to the actual distance, based on the U map data; and
detecting an isolated region corresponding to the object in the real U map, and setting the initial object region based on the isolated region.

11. A computer program product comprising a non-transitory computer-readable recording medium having stored therein a program that causes a computer to execute a method, comprising:
setting an initial object region within image information obtained by capturing an imaging range, the initial object region corresponding to an object existing in the imaging range;
acquiring luminance information indicating luminance in the imaging range; and
adjusting the initial object region to obtain an adjusted object region, based on the acquired luminance information in a luminance detection region set within a lower part of the initial object region, wherein the method further comprises
acquiring distance information indicating a distance between the object and an apparatus equipped with a stereo camera that acquires the image information, and generates look-down view information based on the distance information, wherein the distance information is disparity information;
setting the initial object region based on the look-down view information;
setting the initial object region in a disparity image indicating a distribution of disparities in the imaging range based on the disparity information;
generating U map data indicating a U map, in which a combination of a first axial position, a second axial position, and a disparity in the disparity image is converted to two-dimensional histogram information, based on the disparity information;
generating real U map data indicating a real U map, in which a first axis in the U map is converted to an actual distance and the disparity on a second axis is converted to a decimated disparity value at a decimation rate corresponding to the actual distance, based on the U map data, and
detecting an isolated region corresponding to the object in the real U map, and setting the initial object region based on the isolated region.

Assignees

• Yamada Yuu
• Yoshida Jun
• Ohbayashi Yohichiroh
• Kubozono Hiroki
• Okada Daisuke
• Kida Shintaroh
• Kimura Sukehiro
• Suzuki Tabito
• Ricoh Co Ltd

Inventors

• Yamada Yuu
• Yoshida Jun
• Ohbayashi Yohichiroh
• Kubozono Hiroki
• Okada Daisuke
• Kida Shintaroh
• Kimura Sukehiro
• Suzuki Tabito

Classifications

• G06V2201/07

  Target detection · CPC title

• G06V2201/08

  Detecting or categorising vehicles · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• B60W30/09

  Taking automatic action to avoid collision, e.g. braking and steering · CPC title