Determining distortion by tracking objects across successive frames

Patent metadata

Field	Value
Publication number	US-10539665-B1
Application number	US-201816196618-A
Country	US
Kind code	B1
Filing date	Nov 20, 2018
Priority date	Aug 6, 2018
Publication date	Jan 21, 2020
Grant date	Jan 21, 2020

Abstract

A computer-implemented method of determining relative velocity between a vehicle and an object. The method includes receiving sensor data generated by one or more sensors of the vehicle. The one or more sensors are configured to sense an environment through which the vehicle is moving by following a scan pattern comprising component scan lines. The method also includes obtaining, based on the sensor data and by one or more processors, two or more point cloud frames representative of the environment and tracking, by the one or more processors, a point cloud object across the two or more point cloud frames. Additionally, the method includes determining, based on the tracking and by the one or more processors, a relative velocity of the point cloud object and correcting, by the one or more processors, the point cloud object based on the relative velocity of the point cloud object.

First claim

What is claimed is:

1. A computer-implemented method of correcting point cloud distortion based on tracked object velocity, the method comprising:
receiving sensor data generated by one or more sensors of a vehicle, wherein the one or more sensors are configured to sense an environment through which the vehicle is moving by following a scan pattern comprising component scan lines;
obtaining, based on the sensor data and by one or more processors, two or more point cloud frames representative of the environment;
tracking, by the one or more processors, a point cloud object across the two or more point cloud frames;
determining, based on the tracking and by the one or more processors, a relative velocity of the point cloud object; and
correcting, by the one or more processors, a shape of the point cloud object based on the relative velocity of the point cloud object.

2. The computer-implemented method of claim 1 , wherein correcting the shape of the point cloud object comprises:
applying, by the one or more processors, a machine learning model to determine a correction factor based upon (i) a distance to the point cloud object from the vehicle and (ii) the relative velocity of the point cloud object.

3. The computer-implemented method of claim 2 , further comprising:
applying, by the one or more processors, the correction factor to the point cloud object.

4. The computer-implemented method of claim 1 , further comprising:
identifying, by the one or more processors, a stationary object in the environment of the vehicle;
determining, by the one or more processors, a relative velocity for the stationary object to determine a vehicle velocity; and
based on the relative velocity of point cloud object and the vehicle velocity, determining, by the one or more processors, a velocity of the point cloud object.

5. The computer-implemented method of claim 1 , wherein tracking the point cloud object comprises:
associating, by the one or more processors, the point cloud object with position data based on a first point cloud frame of the two or more point cloud frames.

6. The computer-implemented method of claim 5 , wherein determining the relative velocity of the point cloud object comprises:
comparing, by the one or more processors, position data of the point cloud object based on a second point cloud frame with the position data based on the first point cloud frame.

7. The computer-implemented method of claim 5 , wherein determining the relative velocity of the point cloud object comprises:
comparing, by the one or more processors, a time associated with a second point cloud frame with a time associated with the first point cloud frame.

8. The computer-implemented method of claim 5 , associating the point cloud object with position data comprises:
associating, by the one or more processors, the point cloud object with position data indicative of a particular region of the point cloud object.

9. The computer-implemented method of claim 8 , wherein the particular region is one of a centroid of the point cloud object, a feature of the point cloud object, or a bound of the point cloud object.

10. The computer-implemented method of claim 1 , further comprising:
associating, by the one or more processors, the point cloud object with the determined relative velocity.

11. The computer-implemented method of claim 1 , further comprising:
generating, based on the determined relative velocity of the point cloud object, a predicted future state of the environment of the vehicle.

12. The computer-implemented method of claim 11 , further comprising:
generating, based upon the predicted future state, one or more control signals to control operation of the vehicle.

13. A system within an autonomous vehicle, the system comprising:
a set of sensors configured to generate a set of sensor data by sensing an environment of the vehicle by following a scan pattern comprising component scan lines; and
a computing system configured to:
receive the set of sensor data;
obtain, based on the set of sensor data, two or more point cloud frames representative of the environment;
track a point cloud object across the two or more point cloud frames;
determine, based on the tracking, a relative velocity of the point cloud object; and
correct a shape of the point cloud object based on the relative velocity of the point cloud object.

14. The system of claim 13 , wherein to correct the shape of the point cloud object, the computing system is configured to:
apply a machine learning model to determine a correction factor based upon (i) a distance to the point cloud object from the vehicle and (ii) the relative velocity of the point cloud object.

15. The system of claim 14 , wherein the computing system is configured to:
apply the correction factor to the point cloud object.

16. The system of claim 13 , wherein to correct the point cloud object, the computing system is configured to:
identify a stationary object in the environment of the vehicle; and
determine a relative velocity for the stationary object to determine a vehicle velocity; and
based on the relative velocity of point cloud object and the vehicle velocity, determine a velocity of the point cloud object.

17. The system of claim 13 , wherein to track the point cloud object, the computing system is configured to:
associate the point cloud object with position data based on a first point cloud frame of the two or more point cloud frames.

18. The system of claim 17 , wherein to determine the relative velocity of the point cloud object, the computing system is configured to:
compare position data of the point cloud object based on a second point cloud frame with the position data based on the first point cloud frame.

19. The system of claim 17 , wherein to determine the relative velocity of the point cloud object, the computing system is configured to:
compare a time associated with a second point cloud frame with a time associated with the first point cloud frame.

20. The system of claim 17 , to associate the point cloud object with position data, the computing system is configured to:
associate the point cloud object with position data indicative of a particular region of the point cloud object.

21. The system of claim 20 , wherein the particular region is one of a centroid of the point cloud object, a feature of the point cloud object, or a bound of the point cloud object.

22. The computer-implemented method of claim 13 , wherein the computing system is configured to:
associate the point cloud object with the determined relative velocity.

23. The system of claim 13 , wherein the computing system is configured to:
generate, based on the determined relative velocity of the point cloud object, a predicted future state of the environment of the vehicle.

24. The system of claim 23 , wherein the computing system is configured to:
generate, based upon the predicted future state, one or more control signals to control operation of the vehicle.

Assignees

• Luminar Tech Inc

Inventors

• Danziger Eric C
• Russell Austin K

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/30168

  Image quality inspection · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/75

  involving models · CPC title

• G06T7/73

  using feature-based methods · CPC title