Technology to generalize safe driving experiences for automated vehicle behavior prediction

Patent metadata

Field	Value
Publication number	US-11702105-B2
Application number	US-202016914298-A
Country	US
Kind code	B2
Filing date	Jun 27, 2020
Priority date	Jun 27, 2020
Publication date	Jul 18, 2023
Grant date	Jul 18, 2023

Abstract

Systems, apparatuses and methods may provide for technology that generates, via a first neural network such as a grid network, a first vector representing a prediction of future behavior of an autonomous vehicle based on a current vehicle position and a vehicle velocity. The technology may also generate, via a second neural network such as an obstacle network, a second vector representing a prediction of future behavior of an external obstacle based on a current obstacle position and an obstacle velocity, and determine, via a third neural network such as a place network, a future trajectory for the vehicle based on the first vector and the second vector, the future trajectory representing a sequence of planned future behaviors for the vehicle. The technology may also issue actuation commands to navigate the autonomous vehicle based on the future trajectory for the vehicle.

First claim

We claim:

1. A vehicle computing system comprising:
a sensor interface to obtain data for a vehicle and data for one or more external obstacles; and
a processor coupled to the sensor interface, the processor including one or more substrates and logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic coupled to the one or more substrates to:
generate, via a first neural network, a first vector representing a prediction of future behavior of the vehicle based on a current vehicle position and a vehicle velocity, wherein the first neural network is to produce predicted place cell activations for the vehicle and the first vector represents a probability of vehicle place occupation;
generate, via a second neural network, a second vector representing a prediction of future behavior of an external obstacle based on a current obstacle position and an obstacle velocity, wherein the second neural network is to produce predicted place cell activations for the external obstacle and the second vector represents a probability of obstacle place occupation; and
determine, via a third neural network, a future planned trajectory for the vehicle based on the first vector, the second vector and a route planning input, the future planned trajectory representing a sequence of planned future behaviors for the vehicle.

2. The system of claim 1 , wherein the first neural network comprises a long short-term (LSTM) network, wherein the second neural network comprises a LSTM network, and wherein the third neural network comprises a LSTM network that is to produce a future action sequence for the vehicle.

3. The system of claim 2 , wherein to generate the first vector is further based on a road geometry and an environmental state, wherein to generate the second vector is further based on the road geometry and the environmental state, and wherein to determine the future trajectory for the vehicle is further based on the road geometry, the environmental state, a safety feedback provided by a safety monitor and a destination goal provided by a vehicle route planner.

4. The system of claim 3 , wherein the logic is further to issue actuation commands to navigate the vehicle based on the future trajectory for the vehicle.

5. The system of claim 4 , wherein the logic further comprises a safety component to:
monitor the actuation commands to navigate the vehicle;
restrict one or more of the actuation commands to moderate an unsafe condition; and
provide information for the safety feedback.

6. The system of claim 4 , wherein one or more of a limited set of road geometries or a set of minimum distance safety metrics is used to train the system.

7. A semiconductor apparatus comprising:
one or more substrates; and
logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic coupled to the one or more substrates to:
generate, via a first neural network, a first vector representing a prediction of future behavior of a vehicle based on a current vehicle position and a vehicle velocity, wherein the first neural network is to produce predicted place cell activations for the vehicle and the first vector represents a probability of vehicle place occupation;
generate, via a second neural network, a second vector representing a prediction of future behavior of an external obstacle based on a current obstacle position and an obstacle velocity, wherein the second neural network is to produce predicted place cell activations for the external obstacle and the second vector represents a probability of obstacle place occupation; and
determine, via a third neural network, a future planned trajectory for the vehicle based on the first vector, the second vector and a route planning input, the future planned trajectory representing a sequence of planned future behaviors for the vehicle.

8. The semiconductor apparatus of claim 7 , wherein the first neural network comprises a long short-term (LSTM) network, wherein the second neural network comprises a LSTM network, and wherein the third neural network comprises a LSTM network that is to produce a future action sequence for the vehicle.

9. The semiconductor apparatus of claim 8 , wherein to generate the first vector is further based on a road geometry and an environmental state, wherein to generate the second vector is further based on the road geometry and the environmental state, and wherein to determine the future trajectory for the vehicle is further based on the road geometry, the environmental state, a safety feedback provided by a safety monitor and a destination goal provided by a vehicle route planner.

10. The semiconductor apparatus of claim 9 , wherein the logic is further to issue actuation commands to navigate the vehicle based on the future trajectory for the vehicle.

11. The semiconductor apparatus of claim 10 , wherein the logic further comprises a safety component to:
monitor the actuation commands to navigate the vehicle;
restrict one or more of the actuation commands to moderate an unsafe condition; and
provide information for the safety feedback.

12. The semiconductor apparatus of claim 10 , wherein one or more of a limited set of road geometries or a set of minimum distance safety metrics is used to train the first neural network, the second neural network, and the third neural network together as a unit.

13. The semiconductor apparatus of claim 7 , wherein the logic coupled to the one or more substrates includes transistor channel regions that are positioned within the one or more substrates.

14. At least one non-transitory computer readable storage medium comprising a set of instructions which, when executed by a computing system, cause the computing system to:
generate, via a first neural network, a first vector representing a prediction of future behavior of a vehicle based on a current vehicle position and a vehicle velocity, wherein the first neural network is to produce predicted place cell activations for the vehicle and the first vector represents a probability of vehicle place occupation;
generate, via a second neural network, a second vector representing a prediction of future behavior of an external obstacle based on a current obstacle position and an obstacle velocity wherein the second neural network is to produce predicted place cell activations for the external obstacle and the second vector represents a probability of obstacle place occupation; and
determine, via a third neural network, a future planned trajectory for the vehicle based on the first vector, the second vector and a route planning input, the future planned trajectory representing a sequence of planned future behaviors for the vehicle.

15. The at least one non-transitory computer readable storage medium of claim 14 , wherein the first neural network comprises a long short-term (LSTM) network, wherein the second neural network comprises a LSTM network, and wherein the third neural network comprises a LSTM network that is to produce a future action sequence for the vehicle.

16. The at least one non-transitory computer readable storage medium of claim 15 , wherein to generate the first vector is further based on a road geometry and an environmental state, wherein to generate the second vector is further based on the road geometry and the environmental state, and wherein to determine the future trajectory for the vehicle is further based on the road geometry,

Assignees

• Intel Corp

Inventors

• Alvarez Ignacio J
• Vo Vy
• Felip Leon Javier
• Perez-Ramirez Javier
• Turek Javier Sebastian
• Tepper Mariano
• Gonzalez Aguirre David Israel

Classifications

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• B60W60/0015 Primary

  specially adapted for safety · CPC title

• B60W30/09

  Taking automatic action to avoid collision, e.g. braking and steering · CPC title