Generation and Update of HD Maps Using Data from Heterogeneous Sources

Patent metadata

Field	Value
Publication number	US-2019147331-A1
Application number	US-201715811489-A
Country	US
Kind code	A1
Filing date	Nov 13, 2017
Priority date	Nov 13, 2017
Publication date	May 16, 2019
Grant date	—

Abstract

In one embodiment, a method includes a computing system accessing a training sample that includes first sensor data obtained using a first sensor at a first geographic location, and first metadata comprising information relating to the first sensor. The system may train a machine-learning model by, e.g., generating first map data by processing the training sample using the model and updating the model based on the generated first map data and target map data associated with the first geographic location. The system may then access second sensor data and second metadata, where the second sensor data is obtained using a second sensor that is of the same type as the first sensor. The system may generate second map data associated with a second geographic location by processing the second sensor data and the second metadata using the trained model. A high-definition map may be generated using the second map data.

First claim

What is claimed is:

1 . A method comprising, by a computing system:
accessing a training sample from a training data set, the training sample comprising (1) first sensor data obtained using a first sensor of a first vehicle at a first geographic location and (2) first metadata comprising information relating to the first sensor; accessing target map data associated with the first geographic location; training a machine-learning model, the training comprising:
generating first map data by processing the training sample using the machine-learning model; and
updating the machine-learning model based on the generated first map data and the target map data;
accessing (1) second sensor data obtained using a second sensor of a second vehicle at a second geographic location and (2) second metadata comprising information relating to the second sensor, the second sensor and the first sensor being of a same type of sensor; generating second map data associated with the second geographic location by processing the second sensor data and the second metadata using the trained machine-learning model; and generating a high-definition map using the second map data.

2 . The method of claim 1 , wherein the machine-learning model comprises a convolutional neural network and a deconvolutional neural network, wherein an output of the convolutional neural network is configured to be an input of the deconvolutional neural network.

3 . The method of claim 2 , wherein the generating of the first map data comprises:
generating a latent representation of the training sample by processing the training sample using the convolutional neural network; and generating the first map data by processing the latent representation using the deconvolutional neural network.

4 . The method of claim 2 ,
wherein the machine-learning model comprises a second convolutional neural network; wherein an output of the second convolution neural network is configured to be a second input of the deconvolutional neural network; wherein the training sample further comprises third sensor data obtained using a third sensor of the first vehicle at the first geographic location; wherein the generating of the first map data comprises:
generating a latent representation of the training sample by (1) processing the first sensor data and the first metadata using the convolutional neural network and (2) processing the third sensor data using the second convolutional neural network;
generating the first map data by processing the latent representation using the deconvolutional neural network.

5 . The method of claim 1 , wherein the training sample further comprises (1) third sensor data obtained using a third sensor of the first vehicle at the first geographic location and (2) third metadata comprising information relating to the third sensor, the third sensor and the first sensor having different sensor types.

6 . The method of claim 1 , wherein the first metadata comprises at least one of a spatial location of the first sensor relative to a reference point, an orientation of the first sensor, a capability of the first sensor, a configuration of the first sensor, a model of the first sensor, or a type of the first sensor.

7 . The method of claim 1 , wherein the training sample further comprises environmental data associated with a time at which the first sensor data were obtained using the first sensor, the environmental data comprising at least one of wind speed, temperature, precipitation, time of day, lighting condition, or visibility condition.

8 . The method of claim 1 , wherein the generating of the first map data further comprises processing an object classification using the machine-learning model, wherein the object classification identifies a type of object detected in the first sensor data.

9 . The method of claim 8 , further comprising:
processing the first sensor data using an object classifier to determine the object classification.

10 . The method of claim 1 , further comprising:
transmitting the high-definition map to a plurality of autonomous vehicles, wherein the high-definition map is configured to be used by the plurality of autonomous vehicles for autonomous driving.

11 . The method of claim 1 , further comprising:
accessing a second training sample from the training data set, the second training sample comprising (1) third sensor data obtained using a third sensor of a third vehicle at a third geographic location and (2) third metadata comprising information relating to the third sensor, the third sensor and the first sensor being of the same type of sensor, and the third metadata and the first metadata being different; and accessing second target map data associated with the third geographic location; wherein the training of the machine-learning model further comprises using the second training sample and the second target map data.

12 . A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising:
accessing a training sample from a training data set, the training sample comprising (1) first sensor data obtained using a first sensor of a first vehicle at a first geographic location and (2) first metadata comprising information relating to the first sensor; accessing target map data associated with the first geographic location; training a machine-learning model, comprising:
generating first map data by processing the training sample using the machine-learning model; and
updating the machine-learning model based on the generated first map data and the target map data;
accessing (1) second sensor data obtained using a second sensor of a second vehicle at a second geographic location and (2) second metadata comprising information relating to the second sensor, the second sensor and the first sensor being of a same type of sensor; generating second map data associated with the second geographic location by processing the second sensor data and the second metadata using the trained machine-learning model; and generating a high-definition map using the second map data.

13 . The system of claim 12 , wherein the machine-learning model comprises a convolutional neural network and a deconvolutional neural network, wherein an output of the convolutional neural network is configured to be an input of the deconvolutional neural network.

14 . The system of claim 13 , wherein the generating of the first map data comprises:
generating a latent representation of the training sample by processing the training sample using the convolutional neural network; and generating the first map data by processing the latent representation using the deconvolutional neural network.

15 . The system of claim 13 ,
wherein the machine-learning model comprises a second convolutional neural network; wherein an output of the second convolution neural network is configured to be a second input of the deconvolutional neural network; wherein the training sample further comprises third sensor data obtained using a third sensor of the first vehicle at the first geographic location; wherein the generating of the first map data comprises:
generating a latent representation of the training sample by (1) processing the first sensor data and the first metadata using the convolutional neural network and (2) processing the third sensor data using the

Assignees

• Lyft Inc

Inventors

• Arditi Gil

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title