Systems and methods for streaming processing for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11760280-B2
Application number	US-201815983491-A
Country	US
Kind code	B2
Filing date	May 18, 2018
Priority date	Jan 12, 2018
Publication date	Sep 19, 2023
Grant date	Sep 19, 2023

Abstract

Generally, the present disclosure is directed to systems and methods for streaming processing within one or more systems of an autonomy computing system. When an update for a particular object or region of interest is received by a given system, the system can control transmission of data associated with the update as well as a determination of other aspects by the given system. For example, the system can determine based on a received update for a particular aspect and a priority classification and/or interaction classification determined for that aspect whether data associated with the update should be transmitted to a subsequent system before waiting for other updates to arrive.

First claim

What is claimed is:

1. A computing system, comprising:
a sensor system comprising at least one sensor configured to obtain sensor data at a plurality of consecutive cycles, the sensor data including information descriptive of an environment surrounding an autonomous vehicle;
a memory device for storing the sensor data as the sensor data becomes available from the at least one sensor; and
a data streaming controller comprising one or more processors and one or more non-transitory computer-readable media that store instructions that are executable by the one or more processors to cause the data streaming controller to perform operations, the operations comprising:
for a particular cycle of the plurality of consecutive cycles, classifying a first region within the sensor data as having a first priority level and a second region within the sensor data as having a second priority level, wherein the first priority level is higher than the second priority level, wherein a cycle size associated with the particular cycle corresponds to a 360 degree scene of the environment, and wherein a subset size of the first region corresponds to a portion of the cycle size that is less than 360 degrees;
initiating transfer of the first region of sensor data, from the memory device to a perception system within the computing system upon relay of the first region of sensor data from the at least one sensor to the memory device and before the entire 360 degree scene of the particular cycle is obtained and relayed from the at least one sensor to the memory device and before initiating transfer of the second region of sensor data from the memory device to the perception system in the particular cycle; and
a motion planning system that determines a motion plan for controlling motion of the autonomous vehicle based at least in part on the sensor data transferred to the perception system.

2. The computing system of claim 1 , wherein:
the memory device comprises a rolling buffer to which the sensor data is written at a plurality of successive increments of the sensor data within the particular cycle.

3. The computing system of claim 2 , wherein initiating transfer of the sensor data from the memory device to the perception system within the computing system comprises initiating transfer of respective successive increments of the sensor data within the particular cycle as soon as the respective successive increments are written to the rolling buffer.

4. The computing system of claim 2 , wherein the respective successive increments of the sensor data within the particular cycle comprise an angular slice of a 360 degree scene of the environment surrounding the autonomous vehicle.

5. The computing system of claim 1 , wherein the at least one sensor comprises a camera, a Light Detection and Ranging (LIDAR) system sensor, or a Radio Detection and Ranging (RADAR) system sensor.

6. The computing system of claim 1 , further comprising a priority classification system comprising one or more processors, wherein the priority classification system is configured to, for the particular cycle, determine the first region within the sensor data to classify as having the first priority level and the second region within the sensor data to classify as having the second priority level.

7. The computing system of claim 6 , wherein the first region is determined based on an expected location of an object within a second cycle of the sensor data, the expected location of the object being determined by analyzing a first cycle of the sensor data, wherein the second cycle of the sensor data is obtained subsequently to the first cycle of the sensor data.

8. The computing system of claim 7 , wherein the first region comprises an angular slice of a 360 degree scene of the environment surrounding the autonomous vehicle, the angular slice being inclusive of the object.

9. The computing system of claim 6 , wherein the first region is determined based on a predicted future location of the autonomous vehicle as determined by the motion planning system.

10. The computing system of claim 7 , wherein the priority classification system is further configured to determine a third region within the sensor data to classify as having the first priority level for a third cycle of the sensor data, and wherein the first region within the sensor data for the second cycle at least partially overlaps the third region within the sensor data for the third cycle.

11. The computing system of claim 6 , wherein:
the priority classification system is configured to classify respective portions of the sensor data at the particular cycle as having a particular priority classification; and
the data streaming controller is configured to initiate transfer of the respective portions of the sensor data at the particular cycle based at least in part on the particular priority classification for the respective portions.

12. The computing system of claim 11 , wherein the particular priority classification for the respective portions of the sensor data at the particular cycle comprises either the first priority level or the second priority level.

13. The computing system of claim 1 , further comprising a perception system configured to receive the first region of the sensor data at the particular cycle transferred from the memory device and to generate state data descriptive of at least a current state of one or more objects that are perceived within the sensor data.

14. A computer-implemented method, comprising:
obtaining sensor data at a plurality of consecutive cycles, the sensor data including information descriptive of an environment surrounding an autonomous vehicle;
storing the sensor data in a memory device as the sensor data becomes available from at least one sensor;
for a particular cycle of the plurality of consecutive cycles, classifying a first region within the sensor data corresponding to a first priority level and a second region within the sensor data corresponding to a second priority level, wherein the first priority level is higher than the second priority level, wherein a cycle size associated with the particular cycle corresponds to a 360 degree scene of the environment, and wherein a subset size of the first region corresponds to a portion of the cycle size that is less than 360 degrees;
initiating transfer of the first region of sensor data from the memory device to a perception system upon relay of the first region of sensor data from the at least one sensor to the memory device and before the entire 360 degree scene of the particular cycle is obtained and relayed from the at least one sensor to the memory device and before initiating transfer of the second region of sensor data from the memory device to the perception system in the particular cycle; and
determining a motion plan for controlling motion of the autonomous vehicle based at least in part on the sensor data transferred to the perception system.

15. The computer-implemented method of claim 14 , further comprising determining the first region within the sensor data to classify as having the first priority level and the second region within the sensor data to classify as having the second priority level, based on an expected location of an object within a second cycle of the sensor data, the expected location of the object being determined by analyzing a first cycle of the sensor data, wherein the second cycle of the sensor data is obtained subsequently to the first cycle of the sensor data.

16. The computer-implemented method of claim 14 , further comprising classifying respective portions of the sensor dat

Assignees

• Uatc Llc

Inventors

• Bradley David Mcallister
• Haynes Galen Clark

Classifications

• B60W60/00276

  for two or more other traffic participants · CPC title

• B60R16/023 Primary

  for transmission of signals between vehicle parts or subsystems · CPC title

• G01S17/66

  Tracking systems using electromagnetic waves other than radio waves · CPC title

• G01S17/933

  of aircraft or spacecraft · CPC title

• G01S13/723

  by using numerical data · CPC title