Autonomous vehicle localization using passive image data

Patent metadata

Field	Value
Publication number	US-10474162-B2
Application number	US-201715640334-A
Country	US
Kind code	B2
Filing date	Jun 30, 2017
Priority date	Jul 1, 2016
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

A system to use submaps to control operation of a vehicle is disclosed. A storage system may be provided with a vehicle to store a collection of submaps that represent a geographic area where the vehicle may be driven. A programmatic interface may be provided to receive submaps and submap updates independently of other submaps.

First claim

What is claimed is:

1. A method for performing localization for a vehicle in motion, the method being implemented by one or more processors and comprising:
obtaining a set of sensor data captured by the vehicle at a given instance as the vehicle traverses a given area of a road network, the sensor data including passive image data;
selecting a feature from a known feature set for the given area of the road network based on one or more conditions that affect an appearance of individual features of the known feature set;
matching an image subset of the passive image data to the feature from the known feature set for the given area of the road network; and
determining a location of the vehicle within the given area based on reference location information previously associated with the known feature set.

2. The method of claim 1 , wherein matching the image subset includes detecting an object depicted by the image subset as corresponding to the feature of the known feature set.

3. The method of claim 2 , wherein determining the location is based on determining a difference between a size of the object depicted by the image subset and a dimension associated with the feature when the feature is perceived from a given location.

4. The method of claim 2 , wherein determining the location is based on determining a difference in perspective between the object depicted by the image subset and a perspective associated with the feature when the feature is perceived from a given location.

5. The method of claim 2 , wherein matching the image subset is based on a manipulation of the image subset so that the object depicted by the image subset more closely matches the feature of the known feature set.

6. The method of claim 1 , further comprising determining an orientation of the vehicle relative to a direction of travel as the vehicle progresses through the given area.

7. The method of claim 1 , wherein the one or more conditions are based on at least one of weather, season, or time of day.

8. The method of claim 7 , wherein the one or more conditions include at least one of (i) a presence of precipitation, or (ii) a presence of snow.

9. The method of claim 1 , further comprising identifying a geometry of individual features in the set of features, and wherein selecting the feature is based at least in part on the geometry of the feature.

10. The method of claim 9 , wherein selecting the feature includes selecting features that have a vertical geometry when the one or more conditions correspond to a presence of precipitation or snow.

11. The method of claim 1 , further comprising:
determining a submap from a collection of submaps that corresponds to the area of the road network where the vehicle is located, wherein the submap includes one or more data layers which include the known feature set.

12. The method of claim 11 , wherein the one or more data layers provide a three-dimensional cloud of points, in which individual points of the cloud are associated with imagelets representing one or more features of the known feature set.

13. The method of claim 12 , wherein individual points of the cloud of points are associated with reference information indicating at least one of a location or perspective of a sensor device used to capture an associated imagelet at a previous instance in time.

14. The method of claim 12 , further comprising selecting a set of points with associated imagelets from a larger number of points of the three-dimensional cloud of points with associated imagelets based on information associated with individual points in the larger number of points, the associated imagelets of the selected set of points corresponding to the known feature set, and wherein the information associated with individual points in the larger number of points include at least one of a semantic label, geometric information and/or contextual information.

15. A method for performing localization for a vehicle in motion, the method being implemented by one or more processors and comprising:
obtaining a set of sensor data captured by the vehicle at a given instance as the vehicle traverses a given area of a road network, the sensor data including passive image data;
matching an image subset of the passive image data to a feature from a known feature set for the given area of the road network, wherein matching the image subset includes implementing one or more rules based on a presence of one or more conditions that affect an appearance of individual features of the known feature set; and
determining a location of the vehicle within the given area based on reference location information previously associated with the known feature set.

16. The method of claim 15 , wherein implementing the one or more rules includes avoiding using white pixels of the passive image set for the image subset when snow is present in the given area.

17. The method of claim 15 , wherein implementing the one or more rules includes avoiding using image data of the passive image set which is below a horizon of a camera of the vehicle used to capture the passive image set when the snow is present in the given area.

18. A computer system comprising:
a memory to store a set of instructions;
one or more processors to use the set of instructions to:
obtain a set of sensor data captured by a vehicle at a given instance as the vehicle traverses a given area of a road network, the sensor data including passive image data;
select a feature from a known feature set for the given area of the road network based on one or more conditions that affect an appearance of individual features of the known feature set;
match an image subset of the passive image data to the feature from the known feature set for the given area of the road network; and
determines a location of the vehicle within the given area based on reference location information previously associated with the known feature set.

19. The computer system of claim 18 , wherein the computer system is provided on an autonomous vehicle.

20. The computer system of claim 18 , wherein the computer system communicates with an autonomous vehicle over a network.

21. A computer system comprising:
a memory to store a set of instructions;
one or more processors to use the set of instructions to:
obtain a set of sensor data captured by a vehicle at a given instance as the vehicle traverses a given area of a road network, the sensor data including passive image data;
match an image subset of the passive image data to the feature from a known feature set for the given area of the road network;
determine a location of the vehicle within the given area based on reference location information previously associated with the known feature set;
determine a submap from a collection of submaps that corresponds to the area of the road network where the vehicle is located, wherein the submap includes one or more data layers which include the known feature set, wherein the one or more data layers provide a three-dimensional pointcloud, in which individual points of the pointcloud are associated with imagelets representing one or more features of the known feature set; and
select a set of points with associated imagelets from a larger number of points of the pointcloud with associated imagelets based on information associated with individual points in the larger number of points, the associated imagelets of the selected set of points corresponding to the known feature set, and wherein the information associated with in

Assignees

• Uatc Llc

Inventors

• Browning Brett
• Hansen Peter
• Eade Ethan
• Prasser David
• Larose David
• Zlot Robert

Classifications

• G01C21/3815 Primary

  Road data · CPC title

• G06T7/33

  using feature-based methods · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G01C21/28

  with correlation of data from several navigational instruments · CPC title