Generating environmental input encoding for training neural networks

Patent metadata

Field	Value
Publication number	US-11537134-B1
Application number	US-201815988929-A
Country	US
Kind code	B1
Filing date	May 24, 2018
Priority date	May 25, 2017
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

An encoding of an environment for operating vehicles is obtained, comprising a combination of at least a representation of moving entities with a graph representation of static infrastructure elements. Using the encoding and a set of one or more observations of the environment state, a machine learning model is trained to produce a probabilistic representation of a set of predicted states of the environment. A trained version of the machine learning model is stored and deployed at one or more vehicles to help plan and control vehicle movements.

First claim

What is claimed is:

1. A method, comprising: performing, at one or more computing devices:
obtaining a first encoding of a vehicle operation environment, wherein the first encoding comprises a combination of:
(a) a representation of a plurality of moving entities,
(b) an infrastructure graph, distinct from the representation of the plurality of moving entities, wherein the infrastructure graph comprises a plurality of nodes representing respective static infrastructure components and one or more edges linking the nodes, wherein a particular edge of the one or more edges indicates a relationship between a pair of nodes of the plurality of nodes, and
(c) an occupancy grid map;

training, using at least the first encoding, a first machine learning model to produce a first probabilistic representation of a set of predicted states of the environment, wherein training the first machine learning model comprises:
(a) a first phase of training using the infrastructure graph, wherein at least a portion of the first machine learning model is trained during the first phase to learn representations of one or more infrastructure components; and
(b) a second phase of training, using at least the portion of the first machine learning model resulting from the first phase, to train the first machine learning model to predict actions of the plurality of moving entities using the representation of the plurality of moving entities; and

storing a trained version of the first machine learning model, wherein output produced by the trained version is usable by one or more motion decision making components of a vehicle to generate motion control directives of the vehicle.

2. The method as recited in claim 1 , wherein the first machine learning model comprises a first neural network model that comprises at least a policy model predicting the respective set of actions taken by a plurality of moving entities, and the method further comprising generating a motion-control directive for the vehicle based at least in part on an output of a state transition model predicting at least one or more physical state changes of one or more entities.

3. The method as recited in claim 2 , wherein the state transition model includes a transition model of one or more of: the vehicle, one or more traffic rules, one or more traffic signals, a pedestrian head orientation, or pedestrian pose, and wherein a particular physical state change of the one or more physical state changes includes one or more of: a position, an orientation, a velocity, an acceleration, a current lane, or a traffic light state.

4. The method as recited in claim 1 , wherein the first encoding comprises the occupancy grid map overlaid with the representation of the plurality of moving entities and the infrastructure graph.

5. The method as recited in claim 4 , wherein the occupancy grid map comprises respective representations of one or more of: (a) an obstacle, (b) a member of a set of pre-defined objects, (c) a structured element of the environment, or (d) an unstructured element of the environment.

6. The method as recited in claim 1 , wherein the plurality of nodes includes a first node representing a first static infrastructure component comprising one or more of: (a) a lane segment of a road, (b) an intersection, (c) a traffic sign, (d) a traffic signal, or (e) a pedestrian walkway.

7. The method as recited in claim 6 , wherein the particular edge of the one or more edges indicates one or more of: (a) a geometric constraint associated with a pair of static infrastructure components including the first static infrastructure component, (b) a topological constraint associated with a pair of static infrastructure components, or (c) one or more attributes associated with a pair of static infrastructure components.

8. The method as recited in claim 1 , wherein the representation of the plurality of moving entities comprises one or more of: (a) a padded vector with an upper-bounded capacity, (b) a mapping of respective representations of the moving entities from a source space to a target space with a different dimensionality than the source space, (c) an output of an aggregation layer of a neural network model, or (d) an output obtained from a neural network model comprising one or more recurrent connections.

9. The method as recited in claim 1 , wherein the first probabilistic representation comprises a joint density function.

10. The method as recited in claim 1 , wherein the first probabilistic representation comprises one or more of: (a) a parametric probability density function or (b) a non-parametric probability density function.

11. A system, comprising:
a machine learning model trainer comprising one or more computing devices;
wherein the machine learning model trainer is configured to:
obtain a first encoding of a vehicle operation environment, wherein the first encoding comprises a combination of:
(a) a representation of a plurality of moving entities,
(b) an infrastructure graph, distinct from the representation of the plurality of moving entities, wherein the infrastructure graph comprises a plurality of nodes representing respective infrastructure components and one or more edges linking the nodes, wherein a particular edge of the one or more edges indicates a relationship between a pair of nodes of the plurality of nodes, and
(c) an occupancy grid map;

train, using at least the first encoding, a first machine learning model to produce a first probabilistic representation of a set of predicted states of the environment, wherein training the first machine learning model comprises:
(a) a first phase of training using the infrastructure graph, wherein at least a portion of the first machine learning model is trained during the first phase to learn representations of one or more infrastructure components; and
(b) a second phase of training, using at least the portion of the first machine learning model resulting from the first phase, to train the first machine learning model to predict actions of the plurality of moving entities using the representation of the plurality of moving entities; and

store a trained version of the first machine learning model, wherein output produced by the trained version is usable by one or more motion decision making components of a vehicle to generate motion control directives of the vehicle.

12. The system as recited in claim 11 , wherein the model trainer is configured to train the first machine learning model using one or more observations of the state of the vehicle operation environment.

13. The system as recited in claim 12 , wherein the one or more observations comprise a temporal sequence of observations.

14. The system as recited in claim 11 , wherein the model trainer is configured to train the first machine learning model using one or more latent variables obtained from an estimator, wherein a particular latent variable of the one or more latent variables represents a goal of a moving entity of the plurality of moving entities.

15. The system as recited in claim 11 , wherein at least one computing device of the one or more computing devices is located at a data center, wherein the model trainer is configured to:
cause the trained version of the first machine learning model to be transmitted to the vehicle.

16. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors cause the one or more processors to:
obtain a first encoding of a vehicle operation environment, wherein the first encoding comprises a combination of:
a representation of a plur

Assignees

• Apple Inc

Inventors

• Wiest Juergen
• Goldszmidt Moises
• Gindele Tobias Martin

Classifications

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G05D1/0221 Primary

  involving a learning process · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06N3/0455 Primary

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title