Methods and systems for object detection using laser point clouds

Patent metadata

Field	Value
Publication number	US-9697606-B2
Application number	US-201514749151-A
Country	US
Kind code	B2
Filing date	Jun 24, 2015
Priority date	Apr 25, 2014
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

Methods and systems for object detection using laser point clouds are described herein. In an example implementation, a computing device may receive laser data indicative of a vehicle's environment from a sensor and generate a two dimensional (2D) range image that includes pixels indicative of respective positions of objects in the environment based on the laser data. The computing device may modify the 2D range image to provide values to given pixels that map to portions of objects in the environment lacking laser data, which may involve providing values to the given pixels based on the average value of neighboring pixels positioned by the given pixels. Additionally, the computing device may determine normal vectors of sets of pixels that correspond to surfaces of objects in the environment based on the modified 2D range image and may use the normal vectors to provide object recognition information to systems of the vehicle.

First claim

What is claimed is:

1. A method comprising:
receiving, at a computing device, laser data indicative of an environment of a vehicle, wherein the laser data includes a plurality of data points associated with one or more objects in the environment;
based on the plurality of data points, generating a two dimensional range image that includes a plurality of pixels indicative of respective positions of the one or more objects in the environment relative to the vehicle, wherein generating the two dimensional range image includes providing values to one or more pixels in the two dimensional range image that map to given portions of the environment lacking received laser data;
smoothing, using an integral image process, one or more estimated gradients in the two dimensional range image, wherein the integral image process comprises determining at least one set of average tangential vectors based on respective sums of pixels in the one or more estimated gradients;
determining, using the at least one set of average tangential vectors, one or more cross products of the one or more smoothed gradients;
based on the one or more cross products, determining a plurality of normal vectors of the one or more sets of pixels that correspond to respective surfaces of the one or more objects in the environment; and
based on the plurality of normal vectors of the one or more sets of pixels, providing object recognition information indicative of the one or more objects in the environment to one or more systems of the vehicle.

2. The method of claim 1 , wherein providing values to one or more pixels in the two dimensional range image that map to given portions of the environment lacking received laser data comprises:
determining the values for the one or more pixels based on neighboring pixels having respective values positioned proximal to the one or more pixels in the two dimensional range image.

3. The method of claim 1 , wherein receiving, at the computing device, laser data indicative of the environment of the vehicle comprises:
receiving the plurality of data points associated with one or more objects in the environment in a structured point cloud.

4. The method of claim 1 , wherein receiving, at the computing device, laser data indicative of the environment of the vehicle comprises:
receiving the plurality of data points associated with one or more objects in the environment in an unstructured point cloud.

5. The method of claim 1 , wherein receiving, at the computing device, laser data indicative of the environment of the vehicle comprises:
receiving the plurality of data points associated with one or more objects in the environment in a point cloud format other than a grid-like point cloud.

6. The method of claim 1 , wherein generating the two dimensional range image that includes the plurality of pixels indicative of respective positions of the one or more objects in the environment relative to the vehicle comprises:
generating at least an initial portion of the two dimensional range image upon receiving a first portion of the plurality of data points of a point cloud.

7. The method of claim 1 , wherein providing values to one or more pixels in the two dimensional range image that map to given portions of the environment lacking received laser data is based on an average value of respective neighboring pixels positioned proximal to the one or more pixels in the two dimensional range image.

8. The method of claim 7 , wherein providing values to one or more pixels in the two dimensional range image that map to given portions of the environment lacking received laser data is based on the average value of respective neighboring pixels positioned proximal to the one or more pixels in the two dimensional range image further comprises:
using respective neighboring pixels positioned on top and bottom of a given pixel of the one or more pixels or respective neighboring pixels positioned on left and right of the given pixel of the one or more pixels to estimate the average value.

9. The method of claim 1 , wherein generating the two dimensional range image that includes the plurality of pixels indicative of respective positions of the one or more objects in the environment relative to the vehicle comprises:
projecting the plurality of data points onto a conceptual cylindrical two dimensional surface surrounding the vehicle to generate the two dimensional range image, wherein projecting the plurality of data points onto the conceptual cylindrical two dimensional surface comprises positioning pixels in the two dimensional surface based on data associated with the plurality of data points.

10. The method of claim 1 , wherein the generated two dimensional range image is a virtual 360 degree panorama indicative of the environment of the vehicle.

11. A system comprising:
at least one processor; and
a memory having stored thereon instructions that, upon execution by the at least one processor, cause the system to perform functions comprising:
receiving laser data indicative of an environment of a vehicle, wherein the laser data includes a plurality of data points associated with one or more objects in the environment;
based on the plurality of data points, generating a two dimensional range image that includes a plurality of pixels indicative of respective positions of the one or more objects in the environment relative to the vehicle, wherein generating the two dimensional range image includes providing values to one or more pixels in the two dimensional range image that map to given portions of the environment lacking received laser data;
smoothing, using an integral image process, one or more estimated gradients in the two dimensional range image, wherein the integral image process comprises determining at least one set of average tangential vectors based on respective sums of pixels in the one or more estimated gradients;
determining, using the at least one set of average tangential vectors, one or more cross products of the one or more smoothed gradients;
based on the one or more cross products, determining a plurality of normal vectors of the one or more sets of pixels that correspond to respective surfaces of the one or more objects in the environment; and
based on the plurality of normal vectors of the one or more sets of pixels, providing object recognition information indicative of the one or more objects in the environment to one or more systems of the vehicle.

12. The system of claim 11 , wherein generating the two dimensional range image that includes the plurality of pixels indicative of respective positions of the one or more objects in the environment relative to the vehicle is further based on data provided by a depth camera system of the vehicle.

13. The system of claim 11 , wherein the laser data is provided by a LIDAR module of the vehicle.

14. The system of claim 11 , further comprising:
based on the plurality of normal vectors of the one or more sets of pixels, performing a plane segmentation process to connect the one or more sets of pixels; and
based on performing the plane segmentation process, providing a virtual rendering indicative of the environment to the one or more systems of the vehicle.

15. A non-transitory computer readable medium having stored thereon instructions that, upon execution by a computing device, cause the computing device to perform functions comprising:
receiving laser data indicative of an environment of a vehicle, wherein the laser data includes a plurality of data points associated with one or more objects in the environment;
based on the plurality of data points, generat

Assignees

• Waymo Llc

Inventors

• Stout James
• Zhu Jiajun

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G01S17/931

  of land vehicles · CPC title

• G06T7/521 Primary

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G01S7/41

  using analysis of echo signal for target characterisation; Target signature; Target cross-section · CPC title