Path generation based on predicted actions

Patent metadata

Field	Value
Publication number	US-12420830-B2
Application number	US-202217814497-A
Country	US
Kind code	B2
Filing date	Jul 22, 2022
Priority date	Jul 22, 2022
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

Provided are methods and systems for semantic behavior filtering for prediction improvement. A method for operating an autonomous vehicle, is provided. The method includes obtaining, by at least one processor, semantic image data associated with an environment where an autonomous vehicle is operating. The method includes determining, by the at least one processor, a set of agents in the environment based on the semantic image data. The method includes determining a set of predicted actions for at least one agent of the set of agents. The method includes determining, from the set of predicted actions, a set of secondary predicted actions for the at least one primary agent using semantic data. The method includes determining, from the set of predicted actions, a set of primary predicted actions other than secondary predicted actions The method includes generating a path for the autonomous vehicle based on the set of primary predicted actions.

First claim

What is claimed is:

1. A method for operating an autonomous vehicle, the method comprising:
obtaining, by at least one processor, semantic image data associated with an environment in which the autonomous vehicle is operating, wherein the semantic image data is generated based on images captured by an image sensor, and wherein the semantic image data comprises object attributes associated with objects identified within the images;
determining, by the at least one processor, a set of agents in the environment based on the semantic image data;
determining a set of predicted actions for at least one primary agent of the set of agents, wherein the set of predicted actions are determined using a neural network;
determining, from the set of predicted actions, a set of secondary predicted actions for the at least one primary agent, wherein the set of secondary predicted actions is determined for the at least one primary agent based on a location of the at least one primary agent and based on agent semantic behavior data associated with the at least one primary agent, wherein the agent semantic behavior data comprise logic-based rules and exceptions for predicted agent actions, and wherein a predicted action of the set of predicted actions is determined to be a secondary predicted action of the set of secondary predicted actions based on a probability of an occurrence of the predicted action not satisfying a probability threshold;
determining, from the set of predicted actions, a set of primary predicted actions other than secondary predicted actions, wherein a predicted action of the set of predicted actions is determined to be a primary predicted action of the set of primary predicted actions based on a probability of an occurrence of the predicted action satisfying the probability threshold;
generating a path for the autonomous vehicle based on the set of primary predicted actions; and
providing a control signal to cause the autonomous vehicle to operate along the path for the autonomous vehicle.

2. The method of claim 1 , wherein the set of agents comprise objects configured to move.

3. The method of claim 1 , wherein the set of agents further comprises a secondary agent set, wherein the secondary agent set comprises agents unlikely to interact with the autonomous vehicle.

4. The method of claim 3 , further comprising, transmitting an identity of agents included in the secondary agent set to a processing engine.

5. The method of claim 3 , further comprising, determining a second set of interaction parameters based at least in part on the secondary agent set.

6. The method of claim 1 , wherein the set of agents further comprises a secondary agent set, wherein the secondary agent set comprises agents that will unavoidably interact with the autonomous vehicle.

7. The method of claim 1 , wherein the at least one primary agent is a pedestrian on a sidewalk.

8. The method of claim 7 , wherein the pedestrian is following a predetermined path.

9. The method of claim 8 , wherein the pedestrian is determined to be approaching an intersection between the predetermined path of the pedestrian and the path of the autonomous vehicle.

10. The method of claim 9 , wherein the pedestrian is determined to collide with the autonomous vehicle.

11. The method of claim 1 , wherein generating a path for the autonomous vehicle further comprises determining a plurality of alternative paths for the autonomous vehicle.

12. The method of claim 11 , wherein the plurality of alternative paths include an increased velocity during at least a portion of at least one of the plurality of alternative paths.

13. The method of claim 11 , wherein the plurality of alternative paths include a decreased velocity during at least a portion of at least one of the plurality of alternative paths.

14. A system, comprising:
at least one processor, and
at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to:
obtain, by the at least one processor, semantic image data associated with an environment in which an autonomous vehicle is operating, wherein the semantic image data is generated based on images captured by an image sensor, and wherein the semantic image data comprises object attributes associated with objects identified within the images;
determine, by the at least one processor, a set of agents in the environment based on the semantic image data;
determine a set of predicted actions for at least one primary agent of the set of agents, wherein the set of predicted actions are determined using a neural network;
determine, from the set of predicted actions, a set of secondary predicted actions for the at least one primary agent, wherein the set of secondary predicted actions is determined for the at least one primary agent based on a location of the at least one primary agent and based on agent semantic behavior data associated with the at least one primary agent, wherein the agent semantic behavior data comprise logic-based rules and exceptions for predicted agent actions, and wherein a predicted action of the set of predicted actions is determined to be a secondary predicted action of the set of secondary predicted actions based on a probability of an occurrence of the predicted action not satisfying a probability threshold;
determine, from the set of predicted actions, a set of primary predicted actions other than secondary predicted actions, wherein a predicted action of the set of predicted actions is determined to be a primary predicted action of the set of primary predicted actions based on a probability of an occurrence of the predicted action satisfying the probability threshold;
generate a path for the autonomous vehicle based on the set of primary predicted actions; and
provide a control signal to cause the autonomous vehicle to operate along the path for the autonomous vehicle.

15. The system of claim 14 , wherein the set of agents further comprises a secondary agent set, and wherein the secondary agent set comprises agents unlikely to interact with the autonomous vehicle.

16. At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to:
obtain, by the at least one processor, semantic image data associated with an environment in which an autonomous vehicle is operating, wherein the semantic image data is generated based on images captured by an image sensor, and wherein the semantic image data comprises object attributes associated with objects identified within the images;
determine, by the at least one processor, a set of agents in the environment based on the semantic image data;
determine a set of predicted actions for at least one primary agent of the set of agents, wherein the set of predicted actions are determined using a neural network;
determine, from the set of predicted actions, a set of secondary predicted actions for the at least one primary agent, wherein the set of secondary predicted actions is determined for the at least one primary agent based on a location of the at least one primary agent and based on agent semantic behavior data associated with the at least one primary agent, wherein the agent semantic behavior data comprise logic-based rules and exceptions for predicted agent actions, and wherein a predicted action of the set of predicted actions is determined to be a secondary predicted action of the set of secondary predicted actions based on a probability of an occurrence of the predicted action not satisfying a probability

Assignees

• Motional Ad Llc

Inventors

• Omari Sammy
• Gall Kevin C
• Kabzan Juraj
• Andersen Hans
• Cserna Bence
• Pendleton Scott Drew

Classifications

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• B60W50/0097

  Predicting future conditions · CPC title

• B60W40/04

  Traffic conditions · CPC title

• B60W30/146

  Speed limiting · CPC title

• B60W30/0956

  the prediction being responsive to traffic or environmental parameters · CPC title