Vehicle constraint generation

Patent metadata

Field	Value
Publication number	US-12246734-B1
Application number	US-202318115336-A
Country	US
Kind code	B1
Filing date	Feb 28, 2023
Priority date	Feb 28, 2023
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

Techniques are discussed herein for detecting and managing faults occurring within various vehicle systems, to enable the vehicle to respond safely and appropriately to the faults. In examples, a constraint generator may receive sensor data associated with the operation of a vehicle in an environment. The constraint generator may perturb the sensor data to mimic particular sensor faults, and execute simulations to determine performance degradations of the vehicle systems associated with the particular sensor faults. Based on the performance degradations associated with sensor faults, the constraint generator may determine the appropriate constraints with limitations on the operation of the vehicle, and communicate the constraint parameters to the appropriate vehicle systems accordingly.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:
receive data associated with a vehicle operating in an environment, including sensor data associated with a sensor on the vehicle;
determine, based at least in part on the data, a first metric associated with the vehicle operating in the environment;
perturb, as perturbed sensor data, the sensor data;
execute a driving simulation based at least in part on the perturbed sensor data;
determine a second metric associated with operating a simulated vehicle in the driving simulation;
determine, based at least in part on a difference between first metric and the second metric, a constraint comprising a limitation on a parameter associated with the operation of the vehicle; and
transmitting the constraint to an autonomous vehicle configured to be controlled based at least in part on the constraint.

2. The system of claim 1 , wherein perturbing the sensor data comprises:
determining a time interval, within a duration of time associated with the sensor data; and
perturbing a first portion of the sensor data associated with the time interval, including one or more of:
modifying the first portion of the sensor data, based at least in part on a sensor degradation profile, or
removing the first portion of the sensor data from the sensor data.

3. The system of claim 1 , wherein the difference comprises a difference in a probability of detecting an object represented in the sensor data.

4. The system of claim 1 , wherein:
the sensor data is first sensor data associated with a first sensor,
perturbing the sensor data further includes perturbing second sensor data associated with a second sensor, and
the constraint is associated with a concurrent fault of the first sensor and the second sensor.

5. The system of claim 1 , the operations further comprising:
determining at least one of a driving scenario, a driving region, or a driving condition associated with the sensor data,
wherein transmitting the constraint to the autonomous vehicle comprises transmitting:
first data indicating the constraint, and
second data indicating the at least one of the driving scenario, the driving region, or the driving condition.

6. A method comprising:
receiving data associated with operation of a vehicle in an environment;
determining, based at least in part on the data, a first metric associated with operating the vehicle in the environment;
perturbing, as perturbed data, a portion of the data;
executing a simulation based at least in part on the perturbed data;
determining a second metric associated with operation of a simulated vehicle in the simulation;
determining, based at least in part on the first metric and second metric, a constraint associated with operation of a component of the vehicle; and
transmitting the constraint to a vehicle configured to be controlled based at least in part on the constraint.

7. The method of claim 6 , wherein perturbing the portion of the data comprises at least one of:
modifying, based at least in part on a degradation profile, the portion of the data; or
removing, from the data, the portion of the data.

8. The method of claim 6 , wherein:
the portion of the data is a first portion associated with a first sensor,
the data further includes a second portion associated with a second sensor, and
the perturbed data comprises a perturbed first portion of the data and an unperturbed second portion of the data.

9. The method of claim 6 , further comprising:
determining, as one or more of the first metric or the second metric, a probability of failing to detect an object, and
wherein determining the constraint is based at least in part on the probability.

10. The method of claim 6 , wherein determining the first metric comprises:
executing, based at least in part on the data, a second simulation; and
determining the first metric associated with operating the vehicle in the environment, based at least in part on the second simulation.

11. The method of claim 6 , wherein
the first metric comprises a first histogram,
the second metric comprises a second histogram, and
determining the constraint comprises determining a similarity value between the first histogram and the second histogram.

12. The method of claim 6 , wherein perturbing the portion of the data comprises:
determining a first sensor on the vehicle, wherein the first sensor comprises at least one of:
a lidar sensor,
a radar sensor,
a camera,
a depth sensor, or
a time-of-flight sensor; and

perturbing the portion of the data associated with the first sensor.

13. The method of claim 6 , wherein
the portion of the data is associated with a first sensor,
perturbing the data further includes perturbing a second portion of the data associated with a second sensor, and
the constraint is associated with a concurrent fault of the first sensor and the second sensor.

14. The method of claim 6 , further comprising:
determining at least one of a driving scenario, a driving region, or a driving condition associated with the data,
wherein transmitting the constraint to the vehicle comprises transmitting:
first data indicating the constraint, and
second data indicating the at least one of the driving scenario, the driving region, or the driving condition.

15. The method of claim 6 , wherein the constraint comprises at least one of:
a trajectory constraint,
a dynamic constraint,
a power constraint,
an allowed mission constraint,
a pull-over constraint, or
a release constraint.

16. One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:
receiving data associated with operation of a vehicle in an environment;
determining, based at least in part on the data, a first metric associated with operating the vehicle in the environment;
perturbing, as perturbed data, a portion of the data;
executing a simulation based at least in part on the perturbed data;
determining a second metric associated with operation of a simulated vehicle in the simulation;
determining, based at least in part on the first metric and second metric, a constraint associated with operation of a component of the vehicle; and
transmitting the constraint to a vehicle configured to be controlled based at least in part on the constraint.

17. The one or more non-transitory computer-readable media of claim 16 , wherein determining the first metric comprises:
executing, based at least in part on the data, a second simulation; and
determining the first metric associated with operating the vehicle in the environment, based at least in part on the second simulation.

18. The one or more non-transitory computer-readable media of claim 16 , wherein
the first metric comprises a first histogram,
the second metric comprises a second histogram, and
determining the constraint comprises determining a similarity value between the first histogram and the second histogram.

19. The one or more non-transitory computer-readable media of claim 16 , wherein:
the portion of the data is a first portion associated with a first sensor,
the data further includes a second portion associated with a second sensor, and
the perturbed data comprises a pertur

Assignees

• Zoox Inc

Inventors

• Jung Timothy Henry
• Chandrakumar Janakan
• Macgregor Collin
• Munic Joshua Paul
• Romlewski Matthew Ambrose
• Rosekind Aaron
• Ruiz Sandra
• Terefe Nehemia Girma
• Villaume Nathaniel John
• Willis Darrin Sammons
• Yi He

Classifications

• B60W60/0015

  specially adapted for safety · CPC title

• B60W2050/0215

  Sensor drifts or sensor failures · CPC title

• B60W2556/45

  External transmission of data to or from the vehicle · CPC title

• B60W2556/10

  Historical data · CPC title

• B60W50/0205 Primary

  Diagnosing or detecting failures; Failure detection models · CPC title