Voxel Based Ground Plane Estimation and Object Segmentation

Patent metadata

Field	Value
Publication number	US-2020026292-A1
Application number	US-201916584392-A
Country	US
Kind code	A1
Filing date	Sep 26, 2019
Priority date	Jun 14, 2017
Publication date	Jan 23, 2020
Grant date	—

Abstract

Systems, methods, and apparatuses described herein are directed to performing segmentation on voxels representing three-dimensional data to identify static and dynamic objects. LIDAR data may be captured by a perception system for an autonomous vehicle and represented in a voxel space. Operations may include determining a drivable surface by parsing individual voxels to determine an orientation of a surface normal of a planar approximation of the voxelized data relative to a reference direction. Clustering techniques can be used to grow a ground plane including a plurality of locally flat voxels. Ground plane data can be set aside from the voxel space, and the remaining voxels can be clustered to determine objects. Voxel data can be analyzed over time to determine dynamic objects. Segmentation information associated with ground voxels, static object, and dynamic objects can be provided to a tracker and/or planner in conjunction with operating the autonomous vehicle.

First claim

1 - 14 . (canceled)

15 . A system comprising:
one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the system to perform operations comprising:
receiving a lidar dataset representing an environment;
associating the lidar dataset with a voxel space, the voxel space comprising a voxel;
determining, for the voxel of the voxel space, a normal vector associated with data represented by the voxel;
determining, based at least in part on the normal vector associated with the voxel, that the voxel is a ground voxel; and
determining a ground cluster comprising a set of ground voxels, the set of ground voxels comprising the ground voxel.

16 . The system of claim 15 , the operations further comprising:
determining a reference orientation; and determining that an angle between the normal vector and the reference orientation is below a threshold value; wherein determining that the voxel is the ground voxel is further based at least in part on the angle being below the threshold value.

17 . The system of claim 15 , the operations further comprising:
extracting a set of voxels associated with the ground cluster from the voxel space to identify a subset of the voxel space; clustering the set of voxels to determine a cluster of voxels, wherein a first voxel of the set of voxels is adjacent to at least a second voxel of the set of voxels; determining that a number of voxels associated with the cluster of voxels is below a threshold number of voxels; and determining that the cluster of voxels is not an object.

18 . The system of claim 15 , the operations further comprising:
determining a first cluster of first ground voxels; determining a second cluster of second ground voxels; determining a gradient between the first cluster of locally flat voxels and the second cluster; determining that the gradient is below a gradient threshold; and determining, as the ground cluster and based at least in part on the gradient being below the gradient threshold, the first cluster and the second cluster.

19 . The system of claim 15 , the operations further comprising:
generating, based at least in part on the ground cluster representing a drivable surface for an autonomous vehicle, a trajectory for the autonomous vehicle; and controlling, based at least in part on the trajectory, the autonomous vehicle.

20 . The system of claim 19 , the operations further comprising:
determining an occupancy of voxels over a period of time to determine a dynamic object represented in the voxel space; wherein generating the trajectory is further based at least in part on the dynamic object.

21 . A method comprising:
receiving sensor data representing an environment; associating the sensor data with a voxel space, the voxel space comprising a voxel; determining, for the voxel of the voxel space, a normal vector associated with data represented by the voxel; determining, based at least in part on the normal vector associated with the voxel, that the voxel is a ground voxel; and determining a ground cluster comprising a set of ground voxels, the set of ground voxels comprising the ground voxel.

22 . The method of claim 21 , further comprising:
generating, based at least in part on the ground cluster representing a drivable surface for a vehicle, a trajectory for the vehicle; determining, based at least in part on an orientation of the vehicle, a reference orientation; determining that an angle between the normal vector and the reference orientation is within a threshold value; and determining, based at least in part on the angle being below the threshold value, that the voxel is the ground voxel.

23 . The method of claim 21 , wherein the ground voxel is a first ground voxel, and wherein determining the ground cluster comprises:
selecting the first ground voxel as a seed voxel; determining a second ground voxel adjacent to the seed voxel; and associating the seed voxel and the second ground voxel with the ground cluster.

24 . The method of claim 21 , further comprising:
determining a set of voxels associated with the ground cluster from the voxel space to identify a subset of the voxel space; clustering a first group of voxels of the set of voxels to determine a first object, a first voxel of the first group adjacent to a second voxel of the first group; and clustering a second group of voxels of the set of voxels to determine a second object, a third voxel of the second group adjacent to a fourth voxel of the second group; wherein a first individual voxel of the first group is not adjacent to a second individual second voxel of the second group.

25 . The method of claim 21 , further comprising;
determining a first cluster of first ground voxels; determining a second cluster of second ground voxels; determining a gradient between the first cluster and the second cluster; determining that the gradient is below a gradient threshold; and joining, based at least in part on the gradient being below the gradient threshold, the first cluster with the second cluster to represent the ground cluster.

26 . The method of claim 21 , further comprising:
determining an occupancy of voxels over a period of time to determine a dynamic object represented in the voxel space; and generating, based at least in part on the dynamic object represented in the voxel space, a trajectory for a vehicle.

27 . A non-transitory computer-readable medium storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:
receiving sensor data representing an environment; associating the sensor data with a three-dimensional space discretized as a plurality of voxels; determining, for a voxel of the plurality of voxels, a surface normal associated with a portion of the sensor data associated with the voxel; determining, based at least in part on the surface normal associated with the voxel, that the voxel is a ground voxel; and determining, as a ground cluster, a set of ground voxels including the ground voxel.

28 . The non-transitory computer-readable medium of claim 27 , the operations further comprising:
determining a reference direction, the reference direction corresponding to an orientation of a vehicle; and determining that an angle between the surface normal and the reference direction is below a threshold value; wherein determining that the voxel is the ground voxel is further based at least in part on the reference direction and the angle being below the threshold value.

29 . The non-transitory computer-readable medium of claim 27 , wherein the ground cluster is a first cluster of ground voxels, the operations further comprising:
determining a first average height of first data represented in the first cluster; determining a second average height of second data represented in a second cluster of ground voxels; determining a distance between a first representative point associated with a first voxel of the first cluster and a second representative point associated with a second voxel of the second cluster; determining a difference between the first average height and the second average height; determining, based at least in part on the distance and the difference, a gradient; and determining, based at least in part on the gradient being below a gradient threshold, an updated ground cluster comprising the first cluster and at least a portion of the second cluster.

30 . The no

Assignees

• Zoox Inc

Inventors

• Douillard Bertrand Robert
• Das Subhasis
• Wang Zeng
• Anguelov Dragomir Dimitrov
• Levinson Jesse Sol

Classifications

• G01S13/862

  Combination of radar systems with sonar systems · CPC title

• G01S13/931

  of land vehicles · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G01S13/865

  Combination of radar systems with lidar systems · CPC title

• G06T17/05

  Geographic models · CPC title