Lidar-based classification of object movement

Patent metadata

Field	Value
Publication number	US-9110163-B2
Application number	US-201313917821-A
Country	US
Kind code	B2
Filing date	Jun 14, 2013
Priority date	Jun 14, 2013
Publication date	Aug 18, 2015
Grant date	Aug 18, 2015

Abstract

Within machine vision, object movement is often estimated by applying image evaluation techniques to visible light images, utilizing techniques such as perspective and parallax. However, the precision of such techniques may be limited due to visual distortions in the images, such as glare and shadows. Instead, lidar data may be available (e.g., for object avoidance in automated navigation), and may serve as a high-precision data source for such determinations. Respective lidar points of a lidar point cloud may be mapped to voxels of a three-dimensional voxel space, and voxel clusters may be identified as objects. The movement of the lidar points may be classified over time, and the respective objects may be classified as moving or stationary based on the classification of the lidar points associated with the object. This classification may yield precise results, because voxels in three-dimensional voxel space present clearly differentiable statuses when evaluated over time.

First claim

What is claimed is:

1. A method of detecting movement of objects depicted in a lidar point cloud on a device comprising a processor, the method comprising:
executing on the processor instructions configured to:
identify at least one object represented by a voxel cluster of voxels respectively corresponding to lidar points in the lidar point cloud and sharing an adjacency within a three-dimensional voxel space;
for the respective lidar points in the lidar point cloud:
associate the lidar point with a selected object, and
compare a first position of the lidar point at a first time and a second position of the lidar point at a second time to classify the movement of the lidar point according to the selected object; and

for the respective objects, classify the movement of the object according to the movement of the respective lidar points associated with the object.

2. The method of claim 1 , identifying respective objects further comprising: identifying the object represented by a voxel cluster of voxels sharing an adjacency within an adjacency threshold within the three-dimensional voxel space.

3. The method of claim 2 :
a first axis within the three-dimensional voxel space has a first adjacency threshold;
a second axis within the three-dimensional voxel space has a second adjacency threshold that is different from the first adjacency threshold; and
identifying the object further comprising: identifying the object represented by a cluster of voxels respectively sharing, for respective axes within the three-dimensional space, an adjacency within the adjacency threshold of the axis.

4. The method of claim 1 , identifying the at least one object further comprising:
identifying at least two voxel clusters of voxels within the three-dimensional voxel space;
determining that the at least two voxel clusters are connected.

5. The method of claim 4 , determining that the at least two voxel clusters are connected further comprising:
selecting a first voxel cluster; and
performing a nearest neighbor search of at least one other voxel cluster to identify a connection with the first voxel cluster.

6. The method of claim 1 , classifying the movement of respective lidar points further comprising:
classifying a movement of the respective voxels within the three-dimensional space; and
classifying the movement of the lidar point according to a classification of the movement of the voxel mapped to the lidar point.

7. The method of claim 6 , classifying the movement of the respective voxels further comprising:
for respective voxels, measuring an occupancy duration of the voxel over time by at least one lidar point of the lidar point cloud; and
comparing the occupancy duration of the voxel with an occupancy time variance threshold.

8. The method of claim 7 :
measuring the occupancy duration of the voxel further comprising: measuring a standard deviation of the occupancy duration of the voxel; and
comparing the occupancy duration of the voxel further comprising: comparing the standard deviation of the occupancy duration of the voxel with a standard deviation threshold.

9. The method of claim 6 :
the device further comprising a movement classifier trained to select a classification of the movement of the object; and
classifying the movement of the respective objects further comprising: invoking the movement classifier to select the classification of the movement of the object.

10. The method of claim 1 , the instructions further configured to, after classifying the movement of the respective objects, map the lidar points of the lidar point cloud to the respective objects.

11. The method of claim 10 :
the lidar point cloud having a time axis; and
the instructions further configured to track the movement of the respective objects over the time axis of the lidar point cloud.

12. The method of claim 1 , the instructions further configured to, for respective objects, estimate a position and an orientation of the object according to the lidar points of the lidar point cloud that are associated with the object.

13. The method of claim 1 :
the lidar point cloud having a time axis; and
the instructions further configured to, for respective objects, estimate at least one vector of the object over the time axis of the lidar point cloud.

14. The method of claim 13 , the instructions further configured to, for respective lidar points associated with the object, subtract the at least one vector of the object from the lidar point.

15. The method of claim 13 :
the lidar point cloud detected by a vehicle having at least one vehicle vector; and
estimating the at least one vector of the object further comprising: subtracting the at least one vehicle vector of the vehicle from the at least one vector of the object.

16. The method of claim 1 , the instructions further configured to classify the object as an object type according to the movement of the object.

17. The method of claim 16 :
the device further comprising an object type classifier trained to select an object type classification of the object; and
classifying the object type of the object further comprising: invoking the object type classifier to select the object type classification of the object.

18. The method of claim 1 :
the lidar point cloud associated with at least one image; and
the instructions further configured to, for respective objects classified as a moving object, remove the moving object from the at least one image.

19. A system for detecting movement of objects depicted in a lidar point cloud on a device comprising a processor, the system comprising:
an object identifier configured to:
identify at least one object represented by a voxel cluster of voxels respectively corresponding to lidar points in the lidar point cloud and sharing an adjacency within a three-dimensional voxel space; and

an object movement classifier configured to:
for the respective lidar points in the lidar point cloud:
associate the lidar point with a selected object, and
compare a first position of the lidar point at a first time and a second position of the lidar point at a second time to classify the movement of the lidar point according to the selected object; and

for the respective objects, classify the movement of the object according to the movement of the respective lidar points associated with the object.

20. A computer-readable storage medium comprising instructions that, when executed on a processor of a device, cause the device to detect movement of objects depicted in a lidar point cloud by:
identifying at least one object represented by a voxel cluster of voxels respectively corresponding to lidar points in the lidar point cloud and sharing an adjacency within a three-dimensional voxel space;
for the respective lidar points in the lidar point cloud:
associating the lidar point with a selected object, and
comparing a first position of the lidar point at a first time and a second position of the lidar point at a second time to classify the movement of the lidar point according to the selected object; and

for the respective objects, classifying the movement of the object according to the movement of the respective lidar points associated with the object.

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Rogan Aaron Matthew

Classifications

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G01S17/50 Primary

  Systems of measurement based on relative movement of target · CPC title

• G06F18/24147

  Distances to closest patterns, e.g. nearest neighbour classification · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G01S7/4802

  using analysis of echo signal for target characterisation; Target signature; Target cross-section · CPC title