Vehicle operation and/or simulation decision registry

Patent metadata

Field	Value
Publication number	US-12240484-B1
Application number	US-202117184559-A
Country	US
Kind code	B1
Filing date	Feb 24, 2021
Priority date	Feb 24, 2021
Publication date	Mar 4, 2025
Grant date	Mar 4, 2025

Abstract

Tracking perception and/or planning component decisions may comprise generating a data structure in association with an output determined by a component. This data structure, along with one or more data structures generated in association with other outputs generated by the same or different components of the vehicle, may be used to determine a trace that identifies component(s) that determined outputs that affected a particular component's generation of an output. This trace and/or factors identified by the data structure may be used to detect a problem with a component; identify a component or output therefrom that may be causing an error; determine a portion of a component that has been activated and a frequency associated therewith; and/or visualize operation of the vehicle associated with activation of particular components, among additional or alternate uses discussed herein.

First claim

What is claimed is:

1. A method comprising:
receiving data associated with autonomous operation of a vehicle in an environment;
determining, by a first component from among a plurality of components of the vehicle and based at least in part on the data:
(1) a first output of the first component, wherein the first output is associated with the autonomous operation of the vehicle; and
(2) ancillary data identifying a second output of a second component (a) that the first component used to generate the first output and (b) from among one or more outputs received by the first component from one or more of the plurality of components;

populating a first data structure of a plurality of data structures based at least in part on the first output and the ancillary data, wherein another one of the plurality of data structures is associated with the second output;
detecting, based at least in part on the first data structure, a transient determination associated with operating the vehicle, wherein detecting the transient determination is based at least in part on determining that a duration associated with the first output is less than a threshold duration;
determining, based at least in part on the plurality of data structures and the transient determination, a trace representing at least one of a first characterization of factors that influenced the first output of the first component or a second characterization of a manner in which the first output affected the second output of the second component; and
controlling the vehicle based at least in part on the trace.

2. The method of claim 1 , wherein the trace comprises the first characterization, the first characterization indicating a factor upon which the first output is based, wherein the factor indicates a third output of a third component of the vehicle and an interaction of the third output with the first output.

3. The method of claim 1 , further comprising:
receiving an indication that the first output resulted in an error in operation of the vehicle; and
determining, based at least in part on the trace, at least one of a component or a portion of code associated with the error.

4. The method of claim 1 , wherein the first output is associated with a first determination type and a first time and the method further comprises:
determining one or more additional outputs at one or more times subsequent to the first time associated with at least one of the first determination type or the first component;
determining a rate of change associated with a value or state indicated by the first output and the one or more additional outputs;
determining that the rate of change meets or exceeds a threshold;
determining, based at least in part on the trace, a component associated with the rate of change meeting or exceeding the threshold; and
identifying the component in association with an error.

5. The method of claim 1 , wherein the first data structure is associated with a first time, the first output is a first type of determination, and the method further comprises:
receiving a second data structure associated with a third output at a time subsequent to the first time; and
determining, based at least in part on the first data structure and the second data structure, a measure of complexity of the environment;
wherein controlling the vehicle is further based at least in part on the measure of complexity and determining the measure of complexity is based at least in part on at least one of:
a first number of data structures determined at a first time or over a first time period,
a second number of outputs associated with the plurality of the data structures that indicate a first output type determined at a second time or over a second time period,
an average of the first number or the second number at the first time or second time, respectively,
a moving average of the first number or the second number over a portion of the first time period or the second time period, respectively, or
a standard deviation of the first number or the second number over the first time period or the second time period, respectively.

6. The method of claim 1 , wherein the first data structure is associated with a first time, the first output is associated with a first type of determination, and the method further comprises:
receiving an instruction to replay a portion of operation of the vehicle associated with the first type of determination;
identifying a portion of log data associated with the first type of determination, wherein the log data was generated by operation of the vehicle or execution of a simulation; and
displaying, based at least in part on the portion of log data and the first type of determination, a representation of operation of the vehicle associated with the first type of determination.

7. The method of claim 1 , wherein populating the first data structure based at least in part on the first output and the ancillary data comprises populating the first data structure with a factor upon which the first output is based, wherein the factor indicates at least one of:
a value of the second output that was used by the first component to determine the first output;
a first threshold was used to determine the first output, or
an interaction between the second output and a second threshold was used to determine the first output.

8. A system comprising:
one or more processors; and
a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving data associated with operating a vehicle in an environment;
determining, by a first component of the vehicle and based at least in part on the data, a first output, the first output indicating at least one of an operation to be performed by the vehicle, an operation to be performed by a sensor of the vehicle, or a state associated with the environment;
determining, based at least in part on determining the first output, a first data structure associated with the first output, wherein the first data structure indicates a factor upon which the first output is based, wherein the factor indicates at least one of that the first output was determined based at least in part on a second output of a second component of the vehicle from among a set of outputs received by the first component from a set of components or a rule that was enforced associated with the first output;
detecting, based at least in part on at least one of the first data structure, a transient determination associated with operating the vehicle, wherein detecting the transient determination is based at least in part on determining that a duration associated with the first output is less than a threshold duration;
modifying, as an updated component and based at least in part on the first data structure, at least one of the first component or the second component based at least in part on the transient determination; and
transmitting the updated component to the vehicle.

9. The system of claim 8 , wherein the operations further comprise:
receiving an indication that an error occurred in controlling the vehicle; and
determining, based at least in part on the first data structure, a trace indicating at least one of an upstream output of an upstream component upon which the first output is based at least in part, or a downstream component that receives the first output as an input.

10. The system of claim 9 , wherein the operations further comprise:
generating, based at least in part on the trace, a statement identifying at least one of the upstream component or the downstream component; and
presenting the statement in association with a

Assignees

• Zoox Inc

Inventors

• Shah Prateek Chandresh
• Zhang Rick
• Zhong Wentao
• Schwartz Jeremy

Classifications

• B60W60/001 Primary

  Planning or execution of driving tasks · CPC title

• B60W50/045

  Monitoring control system parameters · CPC title

• G06F16/2272

  Management thereof · CPC title

• G05B17/02

  electric · CPC title

• G05B13/0265

  the criterion being a learning criterion · CPC title