Onboard use of scenario description language

Patent metadata

Field	Value
Publication number	US-11892847-B2
Application number	US-202017092723-A
Country	US
Kind code	B2
Filing date	Nov 9, 2020
Priority date	Sep 1, 2017
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

A domain specific language for use in constructing simulations within real environments is described. In an example, a computing device associated with a vehicle can receive, from one or more sensors associated with the vehicle, sensor data associated with an environment within which the vehicle is positioned. In an example, the vehicle can be an autonomous vehicle. The computing device associated with the vehicle can receive simulated data associated with one or more primitives that are to be instantiated as a scenario in the environment. The computing device can merge the sensor data and the simulated data to generate aggregated data and determine a trajectory along which the vehicle is to drive based at least in part on the aggregated data. The computing device can determine instructions for executing the trajectory and can assess the performance of the vehicle based on how the vehicle responds to the scenario.

First claim

What we claim is:

1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions program the one or more processors to perform actions comprising:
while a vehicle is operating in an environment in which the vehicle is positioned:
receiving sensor data associated with an environment;
determining, based at least in part on the sensor data, a state of the vehicle in relation with the environment;
instantiating, based on the state of the vehicle, a simulated object;
determining, by considering the simulated object as a real object in the environment, a trajectory along which the vehicle is to drive in the environment, wherein the simulated object does not correspond to a physical object in the environment; and
controlling the vehicle to follow the trajectory based at least in part on the simulated object not corresponding to the physical object in the environment.

2. The system of claim 1 , wherein the actions further comprise:
receiving instructions defining the simulated object that is to be instantiated as a scenario in the environment in which the vehicle is positioned;
receiving instructions defining at least one primitive corresponding to the simulated object;
receiving a sequence defining an association between individual primitives of the at least one primitive; and
instantiating the at least one primitive, based at least in part on the sensor data, to generate the scenario.

3. The system of claim 2 , wherein the sequence is defined using temporal logic.

4. The system of claim 2 , wherein the actions further comprise determining, based at least in part on execution of the trajectory, information associated with the scenario, the information describing how the vehicle responds given the scenario created in the environment.

5. The system of claim 4 , wherein determining the information associated with the scenario comprises validating that the vehicle performed in accordance with a set of rules.

6. The system of claim 4 , wherein determining the information associated with the scenario comprises at least one of:
determining an operational space of the vehicle given the scenario; or
generating feedback for improving at least one of operations or designs of the vehicle.

7. The system of claim 1 , wherein the vehicle is an autonomous vehicle.

8. The system of claim 1 , wherein the actions further comprise:
determining, based at least in part on the sensor data, a real object in the environment; and
merging the simulated object and the real object into a group of objects,
wherein the simulated object is indistinguishable from the real object for determining the trajectory.

9. A computer-implemented method comprising:
while a vehicle is operating in an environment in which the vehicle is positioned:
generating a scenario including a simulated object, the simulated object based, at least in part, on a primitive;
receiving sensor data associated with a vehicle in an environment;
determining perception data based at least in part on the sensor data;
merging the simulated object and the perception data to generate aggregated data;
determining, based at least in part on the aggregated data, a trajectory along which the vehicle is to drive in the environment by considering the simulated object as a real object in the environment; and
controlling the vehicle to follow the trajectory based at least in part on the simulated object not corresponding to the real object in the environment.

10. The computer-implemented method of claim 9 , further comprising instantiating the scenario based at least in part on the perception data.

11. The computer-implemented method of claim 9 , wherein the sensor data is received from one or more sensors comprising one or more of a LIDAR sensor, a camera sensor, a RADAR sensor, or a SONAR sensor.

12. The computer-implemented method of claim 9 , wherein the vehicle is an autonomous vehicle, and the computer-implemented method further comprises generating one or more signals for causing the vehicle to drive along the trajectory.

13. The computer-implemented method of claim 9 , further comprising evaluating a performance of the vehicle based at least in part on a nominal performance of the vehicle.

14. The computer-implemented method of claim 9 , further comprising evaluating a performance of the vehicle based at least in part on a formula indicating a set of driving rules.

15. One or more non-transitory computer readable media storing computer executable instructions that, when executed, cause one or more processors to perform operations comprising:
while a vehicle is operating in an environment in which the vehicle is positioned:
receiving sensor data associated with an environment of the vehicle;
receiving a primitive;
generating simulated data including a simulated object in the environment, the simulated object based, at least in part, on the primitive;
merging the sensor data and the simulated data to generate aggregated data including the simulated object;
determining, based at least in part on the aggregated data, a trajectory along which the vehicle is to drive; and
controlling the vehicle to follow the trajectory based at least in part on the simulated object not corresponding to any real object in the environment.

16. The one or more non-transitory computer readable media of claim 15 , the operations further comprising:
determining one or more objects in the environment based at least in part on merging the sensor data and the simulated data; and
determining the trajectory based at least in part on the one or more objects.

17. The one or more non-transitory computer readable media of claim 15 , the operations further comprising:
determining, based at least in part on the sensor data, a real object in the environment; and
determining, based at least in part on the simulated data, the simulated object in the environment,
wherein merging the sensor data and the simulated data comprises merging the real object and the simulated object into a group of objects.

18. The one or more non-transitory computer readable media of claim 17 , wherein the real object is indistinguishable from the simulated object for determining the trajectory.

19. The one or more non-transitory computer readable media of claim 15 , wherein the simulated data further comprises a sequence defining an association between individual primitives of one or more primitives.

20. The one or more non-transitory computer readable media of claim 15 , the operations further comprising:
determining a command to cause the vehicle to drive along the trajectory;
executing the command; and
assessing a performance of the vehicle based at least in part on executing the command.

Assignees

• Zoox Inc

Inventors

• Askeland Jacob Lee
• Cahoon Ryan Martin
• Linscott Gary

Classifications

• G05D1/0212 Primary

  with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287) · CPC title

• G05B17/02

  electric · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• B60W60/00276 Primary

  for two or more other traffic participants · CPC title

• B60W2050/0018

  Method for the design of a control system · CPC title