Systems and methods for remote status detection of autonomous vehicles

Patent metadata

Field	Value
Publication number	US-12130620-B2
Application number	US-202318175748-A
Country	US
Kind code	B2
Filing date	Feb 28, 2023
Priority date	May 18, 2018
Publication date	Oct 29, 2024
Grant date	Oct 29, 2024

Abstract

Systems and methods are provided for remotely detecting a status associated with an autonomous vehicle and generating control actions in response to such detections. In one example, a computing system can access a third-party communication associated with an autonomous vehicle. The computing system can determine, based at least in part on the third-party communication, a predetermined identifier associated with the autonomous vehicle. The computing system can determine, based at least in part on the third-party communication, a status associated with the autonomous vehicle, and transmit one or more control messages to the autonomous vehicle based at least in part on the predetermined identifier and the status associated with the autonomous vehicle.

First claim

What is claimed is:

1. A computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that store instructions for execution by the one or more processors to cause the computing system to perform operations, the operations comprising:
(a) determining a particular identifier of an autonomous vehicle based on a third-party communication indicating status information relating to the autonomous vehicle;
(b) communicating a request for sensor data to the autonomous vehicle associated with the particular identifier;
(c) receiving sensor data from a sensor positioned on the autonomous vehicle, wherein the sensor data comprises one or more of imagery or LIDAR data of an external surface of the autonomous vehicle;
(d) processing the sensor data from the autonomous vehicle with a machine-learned model trained to analyze the sensor data to identify a condition of the external surface of the autonomous vehicle and to generate as an output a status associated with the condition of the external surface of the autonomous vehicle based on the sensor data; and
(e) providing one or more control messages to the autonomous vehicle based on the status associated with the condition of the external surface of the autonomous vehicle.

2. The computing system of claim 1 , wherein the machine-learned model trained to analyze the sensor data comprises a neural network.

3. The computing system of claim 2 , wherein the sensor comprises a camera.

4. The computing system of claim 2 , wherein the sensor comprises a LIDAR sensor.

5. The computing system of claim 1 , wherein (c) further comprises receiving, in response to communicating a request for sensor data to the autonomous vehicle, the sensor data from a remote access system associated with the autonomous vehicle.

6. The computing system of claim 1 , wherein the third-party communication includes imagery depicting at least a portion of the autonomous vehicle, the imagery having been captured at a client computing device by a user of the client computing device.

7. The computing system of claim 6 , wherein (d) comprises processing the sensor data from the autonomous vehicle and the imagery from the third-party communication with the machine-learned model trained to analyze both the sensor data and the imagery to identify a condition of an external surface of the autonomous vehicle and to generate as an output a status associated with the condition of the external surface of the autonomous vehicle based on the sensor data and the imagery.

8. The computing system of claim 6 , wherein determining the particular identifier includes detecting the particular identifier based at least in part on an analysis of the imagery.

9. The computing system of claim 1 , wherein the one or more control messages include instructions for the autonomous vehicle to execute a controlled stop in a safe state.

10. The computing system of claim 1 , wherein the one or more control messages include instructions for the autonomous vehicle to proceed toward a predetermined location.

11. A computer-implemented method, comprising:
(a) determining a particular identifier of an autonomous vehicle based on a third-party communication indicating status information relating to the autonomous vehicle;
(b) communicating a request for sensor data to the autonomous vehicle associated with the particular identifier;
(c) receiving sensor data from a sensor positioned on the autonomous vehicle, wherein the sensor data comprises one or more of imagery or LIDAR data of an external surface of the autonomous vehicle;
(d) processing the sensor data from the autonomous vehicle with a machine-learned model trained to analyze the sensor data to identify a condition of the external surface of the autonomous vehicle and to generate as an output a status associated with the condition of the external surface of the autonomous vehicle based on the sensor data; and
(e) providing one or more control messages to the autonomous vehicle based on the status associated with the condition of the external surface of the autonomous vehicle.

12. The computer-implemented method of claim 11 , wherein
the machine-learned model trained to analyze the sensor data from the autonomous vehicle comprises a neural network.

13. The computer-implemented method of claim 12 , wherein the sensor comprises a camera.

14. The computer-implemented method of claim 12 , wherein the sensor comprises a LIDAR sensor.

15. The computer-implemented method of claim 11 , wherein (c) further comprises receiving, in response to communicating a request for sensor data to the autonomous vehicle, the sensor data from a remote access system associated with the autonomous vehicle.

16. The computer-implemented method of claim 11 , wherein the third-party communication includes imagery depicting at least a portion of the autonomous vehicle, the imagery having been captured at a client computing device by a user of the client computing device.

17. The computer-implemented method of claim 16 , wherein (d) comprises processing the sensor data from the autonomous vehicle and the imagery from the third-party communication with the machine-learned model trained to analyze both the sensor data and the imagery to identify a condition of an external surface of the autonomous vehicle and to generate as an output a status associated with the condition of the external surface of the autonomous vehicle based on the sensor data and the imagery.

18. The computer-implemented method of claim 16 , wherein determining the particular identifier includes detecting the particular identifier based at least in part on an analysis of the imagery.

19. The computer-implemented method of claim 11 , wherein the one or more control messages include instructions for the autonomous vehicle to: (i) execute a controlled stop in a safe state; or (ii) proceed toward a predetermined location.

20. One or more tangible, non-transitory computer-readable media storing computer-readable instructions for execution by one or more processors to cause the one or more processors to perform operations, the operations comprising:
(a) determining a particular identifier of an autonomous vehicle based on a third-party communication indicating status information relating to the autonomous vehicle;
(b) communicating a request for sensor data to the autonomous vehicle associated with the particular identifier;
(c) receiving sensor data from a sensor positioned on the autonomous vehicle, wherein the sensor data comprises one or more of imagery or LIDAR data of an external surface of the autonomous vehicle;
(d) processing the sensor data from the autonomous vehicle with a machine-learned model trained to analyze the sensor data to identify a condition of the external surface of the autonomous vehicle and to generate as an output a status associated with the condition of the external surface of the autonomous vehicle based on the sensor data; and
(e) providing one or more control messages to the autonomous vehicle based on the status associated with the condition of the external surface of the autonomous vehicle.

Assignees

• Aurora Operations Inc

Inventors

• Poeppel Scott C
• Zych Noah
• Wood Matthew Shaw
• Vandenberg Iii Dirk John

Classifications

• G06N3/08

  Learning methods · CPC title

• G05D1/226

  Communication links with the remote-control arrangements · CPC title

• G05D1/227

  Handing over between remote control and on-board control; Handing over between remote control arrangements · CPC title

• G07C5/008

  communicating information to a remotely located station (transmission systems for measured values G08C) · CPC title

• G08G1/0175

  by photographing vehicles, e.g. when violating traffic rules · CPC title