Information processing apparatus, information processing method, program, and mobile object

Patent metadata

Field	Value
Publication number	US-11373418-B2
Application number	US-201816634580-A
Country	US
Kind code	B2
Filing date	Jul 25, 2018
Priority date	Aug 4, 2017
Publication date	Jun 28, 2022
Grant date	Jun 28, 2022

Abstract

An information processing apparatus capable of detecting a plane constituting a movement-enabling region. A normal direction of a plane constituting a road surface is detected on the basis of polarized images in multiple polarizing directions acquired by a polarization camera. A laser ranging sensor measures a distance to a point on the road surface so as to measure a position of the point. The plane constituting the road surface is identified on the basis of information regarding the normal direction of the plane constituting the road surface and information regarding the position of the point on the road surface.

First claim

The invention claimed is:

1. An information processing apparatus comprising:
a polarization camera configured to acquire polarized images in a plurality of polarizing directions,
circuitry configured to:
detect a normal direction of a plane constituting a movement-enabling region for a mobile object on a basis of the polarized images;
identify a position of a point on the plane constituting the movement-enabling region; and
recognize the plane constituting the movement-enabling region on a basis of the normal direction of the movement-enabling region and the position of the point in the movement-enabling region,
wherein the detecting the normal direction of the plane constituting the movement-enabling region for the mobile object on the basis of the polarized images comprises:
detecting, in units of pixels, a normal direction of a surface of a subject in the polarized images before outputting the detected normal direction as a normal vector map;
reconstructing an unpolarized image from the polarized images; and
setting an attribute of the subject to each of the pixels in the unpolarized image,

wherein
the circuitry detects a dominant normal direction on the normal vector map corresponding to a region having those pixels in the unpolarized image of which the attribute is set to the movement-enabling region, the dominant normal direction being the normal direction of the plane constituting the movement-enabling region, and

wherein the subject is semantically labelled as a road surface, an obstacle, or sky.

2. The information processing apparatus according to claim 1 , wherein
the circuitry sets, through semantic segmentation, the attribute of the subject to each of the pixels in the unpolarized image.

3. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to generate an environmental map on a basis of information regarding the attribute of the subject set to each of the pixels in the unpolarized image and information regarding the plane identified by the plane identification section.

4. The information processing apparatus according to claim 3 , wherein,
given the information regarding the attribute of the subject set to each of the pixels in the unpolarized image, the circuitry generates the environmental map on a basis of information regarding the pixels of which the attribute is set to the movement-enabling region and information regarding the pixels of which the attribute is set to an obstacle, the environmental map being indicative of a position of the obstacle on the plane constituting the movement-enabling region.

5. The information processing apparatus according to claim 4 , wherein,
given the information regarding the attribute of the subject set to each of the pixels in the unpolarized image, the circuitry generates the environmental map by taking the position on the plane constituting the movement-enabling region and corresponding to a boundary pixel in a vertically downward direction between the region of the pixels of which the attribute is set to the movement-enabling region on one hand and the region of the pixels of which the attribute is set to the obstacle on the other hand, as the position of the obstacle on the plane constituting the movement-enabling region.

6. The information processing apparatus according to claim 3 , wherein
the environmental map is an occupancy grid map.

7. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to integrate a coordinate system indicative of the position of the point identified by the circuitry in the movement-enabling region with a coordinate system of the polarized images acquired by the polarization camera.

8. The information processing apparatus according to claim 1 , further comprising:
a ranger configured to measure a distance to a point in the movement-enabling region, wherein
the circuitry is configured to identify the position of the point in the movement-enabling region on a basis of the distance, measured by the ranger, to the point in the movement-enabling region.

9. The information processing apparatus according to claim 8 , wherein
the ranger includes a laser ranging sensor, a stereo camera, or a millimeter-wave radar.

10. The information processing apparatus according to claim 1 , wherein
the movement-enabling region includes a road surface.

11. An information processing method comprising the steps of:
acquiring polarized images in a plurality of polarizing directions;
detecting, on a basis of the polarized images, a normal direction of a plane constituting a movement-enabling region for a mobile object;
identifying a position of a point in the movement-enabling region; and
recognizing the plane constituting the movement-enabling region on a basis of the normal direction of the movement-enabling region and the position of the point in the movement-enabling region,
wherein the detecting the normal direction of the plane constituting the movement-enabling region for the mobile object on the basis of the polarized images comprises:
detecting, in units of pixels, a normal direction of a surface of a subject in the polarized images before outputting the detected normal direction as a normal vector map;
reconstructing an unpolarized image from the polarized images; and
setting an attribute of the subject to each of the pixels in the unpolarized image, wherein
a dominant normal direction on the normal vector map corresponding to a region having those pixels in the unpolarized image of which the attribute is set to the movement-enabling region is detected, the dominant normal direction being the normal direction of the plane constituting the movement-enabling region, and
wherein the subject is semantically labelled as a road surface, an obstacle, or sky.

12. A non-transitory computer readable memory device that includes instructions that when executed by a processor cause the processor to perform a method, the method comprising:
acquiring polarized images in a plurality of polarizing directions;
detecting, on a basis of the polarized images, a normal direction of a plane constituting a movement-enabling region for a mobile object;
identifying a position of a point on the plane constituting the movement-enabling region; and
recognizing the plane constituting the movement-enabling region on a basis of the normal direction of the movement-enabling region and the position of the point in the movement-enabling region,
wherein the detecting the normal direction of the plane constituting the movement-enabling region for the mobile object on the basis of the polarized images comprises:
detecting, in units of pixels, a normal direction of a surface of a subject in the polarized images before outputting the detected normal direction as a normal vector map;
reconstructing an unpolarized image from the polarized images; and
setting an attribute of the subject to each of the pixels in the unpolarized image, wherein
a dominant normal direction on the normal vector map corresponding to a region having those pixels in the unpolarized image of which the attribute is set to the movement-enabling region is detected, the dominant normal direction being the normal direction of the plane constituting the movement-enabling region, and
wherein the subject is semantically labelled as a road surface, an obstacle, or sky.

13. A mobile object comprising the information processing apparatus of claim 1 .

Assignees

• Sony Corp

Inventors

• Motoyama Takuto
• Oyaizu Hideki
• Sutou Yasuhiro
• Yamazaki Toshio
• Doba Kentaro

Classifications

• G01C21/3602 Primary

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V10/803

  of input or preprocessed data · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title