Teleassistance data prioritization for self-driving vehicles

Patent metadata

Field	Value
Publication number	US-10983520-B2
Application number	US-201916271558-A
Country	US
Kind code	B2
Filing date	Feb 8, 2019
Priority date	Mar 7, 2017
Publication date	Apr 20, 2021
Grant date	Apr 20, 2021

Abstract

A system can analyze a live sensor view of a self-driving vehicle (SDV) in accordance with a safety threshold to detect objects of interest along a route and classify each detected object of interest. When the safety threshold is not met, the system can transmit a teleassistance inquiry using LIDAR data to a backend computing system. When a certainty threshold is not met for an object of interest, the system can transmit a different teleassistance inquiry using image data to the backend computing system.

First claim

What is claimed is:

1. A system comprising:
a network communication interface;
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to:
analyze a live sensor view generated by a set of sensors of a self-driving vehicle (SDV) in accordance with a safety threshold to (i) detect objects of interest along a route, and (ii) classify each respective object of the detected objects of interest;
for a particular object of the detected objects of interest, determine that the safety threshold is not met;
in response to determining that the safety threshold is not met for the particular object, transmit, using the network communication interface, a first teleassistance inquiry to a backend computing system, the first teleassistance inquiry comprising LIDAR data generated by a LIDAR device of the SDV, wherein the set of sensors of the SDV includes sensors of the LIDAR device;
determine that a classification of one or more of the detected objects of interest does not satisfy a certainty threshold;
in response to determining that the classification of the one or more of the detected objects of interest does not satisfy the certainty threshold, transmit, using the network communication interface, a second teleassistance inquiry to the backend computing system, the second teleassistance inquiry comprising image data generated by one or more cameras of the SDV, the image data including the one or more of the detected objects of interest, wherein the set of sensors of the SDV includes sensors of the one or more cameras, and wherein the second teleassistance inquiry is different from the first teleassistance inquiry;
for each of the first and the second teleassistance inquiries, receive a resolution response from the backend computing system; and

provide a control system of the SDV with data corresponding to the resolution response to enable the SDV to proceed in accordance with the resolution response.

2. The system of claim 1 , wherein the safety threshold not being met corresponds to an obstruction in the live sensor view.

3. The system of claim 1 , wherein the executed instructions cause the system to transmit LIDAR data and not image data when the safety threshold is not met.

4. The system of claim 1 , wherein the executed instructions cause the system to transmit image data and not LIDAR data when the certainty threshold is not met.

5. The system of claim 1 , wherein the executed instructions further cause the system to:
monitor bandwidth of each of a plurality of wireless communication modules of the network communication interface to determine a highest bandwidth communication module;
wherein the executed instructions cause the system to transmit each of the first and the second teleassistance inquiries to the backend computing system using the highest bandwidth communication module.

6. The system of claim 5 , wherein the executed instructions further cause the system to:
duplicate transmission of each of the first and the second teleassistance inquiries by transmitting the first and the second teleassistance inquiries over a second communication module from the plurality of wireless communication modules.

7. The system of claim 6 , wherein the highest bandwidth communication module and the second communication module both comprise Long Term Evolution (LTE) communications modules.

8. The system of claim 6 , wherein the highest bandwidth communication module comprises a LTE communication module and the second communication module comprises a Wi-Fi module.

9. The system of claim 1 , wherein the first and the second teleassistance inquiries include telemetry data for the SDV.

10. The system of claim 1 , wherein the first and the second teleassistance inquiries include localization information for the SDV.

11. The system of claim 1 , wherein the first and the second teleassistance inquiries include diagnostics data for the SDV.

12. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
analyze a live sensor view generated by a set of sensors of a self-driving vehicle (SDV) in accordance with a safety threshold to (i) detect objects of interest along a route, and (ii) classify each respective object of the detected objects of interest;
for a particular object of the detected objects of interest, determine that the safety threshold is not met;
in response to determining that the safety threshold is not met for the particular object, transmit, using a network communication interface, a first teleassistance inquiry to a backend computing system, the first teleassistance inquiry comprising LIDAR data generated by a LIDAR device of the SDV, wherein the set of sensors of the SDV includes sensors of the LIDAR device;
determine that a classification of one or more of the detected objects of interest does not satisfy a certainty threshold;
in response to determining that the classification of the one or more of the detected objects of interest does not satisfy the certainty threshold, transmit, using the network communication interface, a second teleassistance inquiry to the backend computing system, the second teleassistance inquiry comprising image data generated by one or more cameras of the SDV, the image data including the one or more detected objects of interest, wherein the set of sensors of the SDV includes sensors of the one or more cameras, and wherein the second teleassistance inquiry is different from the first teleassistance inquiry;
for each of the first and the second teleassistance inquiries, receive a resolution response from the backend computing system; and
provide a control system of the SDV with data corresponding to the resolution response to enable the SDV to proceed in accordance with the resolution response.

13. The non-transitory computer readable medium of claim 12 , wherein the safety threshold not being met corresponds to an obstruction in the live sensor view.

14. The non-transitory computer readable medium of claim 12 , wherein the executed instructions cause the system to transmit LIDAR data and not image data when the safety threshold is not met.

15. The non-transitory computer readable medium of claim 12 , wherein the executed instructions cause the system to transmit image data and not LIDAR data when the certainty threshold is not met.

16. The non-transitory computer readable medium of claim 12 , wherein the executed instructions further cause the system to
monitor bandwidth of each of a plurality of wireless communication modules of the network communication interface to determine a highest bandwidth communication module;
wherein the executed instructions cause the system to transmit each of the first and the second teleassistance inquiries to the backend computing system using the highest bandwidth communication module.

17. The non-transitory computer readable medium of claim 16 , wherein the executed instructions further cause the system to:
duplicate transmission of each of the first and the second teleassistance inquiries by transmitting the first and the second teleassistance inquiries over a second communication module from the plurality of wireless communication modules.

18. The non-transitory computer readable medium of claim 17 , wherein the highest bandwidth communication module and the second communication module both comprise Long Term Evolution (LTE) communications modules.

19. A computer-implemented method performed by one or m

Assignees

• Uatc Llc
• Uber Technologies Inc

Inventors

• Kroop Benjamin
• Ross William
• Heine Andrew

Classifications

• B60W10/18

  including control of braking systems · CPC title

• B60W10/04

  including control of propulsion units · CPC title

• G07C5/008

  communicating information to a remotely located station (transmission systems for measured values G08C) · CPC title

• B60W2554/00

  Input parameters relating to objects · CPC title

• B60W60/0015

  specially adapted for safety · CPC title