Teleassistance data prioritization for self-driving vehicles

Patent metadata

Field	Value
Publication number	US-10293818-B2
Application number	US-201715452431-A
Country	US
Kind code	B2
Filing date	Mar 7, 2017
Priority date	Mar 7, 2017
Publication date	May 21, 2019
Grant date	May 21, 2019

Abstract

A self-driving vehicle can operate by analyzing a live sensor view to autonomously operate acceleration, braking, and steering systems of the SDV along a current route. Based on a detected anomaly associated with the live sensor view, the SDV can transmit a teleassistance inquiry to a backend transport system in accordance with a data prioritization scheme. The SDV can receive a resolution response from the backend transport system based on the teleassistance inquiry, and proceed in accordance with the resolution response.

First claim

What is claimed is:

1. A self-driving vehicle (SDV) comprising:
a communication interface;
a sensor system comprising one or more LIDAR sensors generating LIDAR data and one or more cameras generating image data, the LIDAR data and image data providing a live sensor view of a situational environment of the SDV;
acceleration, braking, and steering systems; and
a control system to execute instructions that cause the control system to:
analyze the live sensor view in accordance with a safety threshold to (i) detect objects of interest along a current route, (ii) classify each of the detected objects of interest in accordance with a certainty threshold, and (iii) autonomously operate the acceleration, braking, and steering systems along the current route;
when the safety threshold is not met, transmit a first teleassistance inquiry to a backend transport system, the first teleassistance inquiry comprising LIDAR data;
when the certainty threshold for an indeterminate object is not met, transmit a second teleassistance inquiry to the backend transport system, the second teleassistance inquiry comprising image data that includes the indeterminate object;
for each of the first and the second teleassistance inquiries, receive a resolution response from the backend transport system; and
autonomously operate the acceleration, braking, and steering systems to proceed in accordance with the resolution response.

2. The SDV of claim 1 , wherein the safety threshold not being met corresponds to an obstruction in the live sensor view.

3. The SDV of claim 1 , wherein the executed instructions cause the control system to transmit LIDAR data and not image data when the safety threshold is not met.

4. The SDV of claim 1 , wherein the executed instructions cause the control system to transmit image data and not LIDAR data when the certainty threshold is not met.

5. The SDV of claim 1 , wherein the communication interface comprises a plurality of wireless communication modules enabling communication with the backend transport system via a plurality of wireless communication providers, and wherein the executed instructions further cause the control system to:
monitor bandwidth of each of the plurality of wireless communication modules to determine a highest bandwidth communication module;
wherein the executed instructions cause the control system to transmit each of the first and the second teleassistance inquiries to the backend transport system using the highest bandwidth communication module.

6. The SDV of claim 5 , wherein the executed instructions further cause the control system to:
duplicate transmission of each of the first and the second teleassistance inquiries by transmitting the first and the second teleassistance inquiries over a second communication module from the plurality of wireless communication modules.

7. The SDV of claim 6 , wherein the highest bandwidth communication module and the second communication module both comprise Long Term Evolution (LTE) communications modules.

8. The SDV of claim 6 , wherein the highest bandwidth communication module comprises a LTE communication module and the second communication module comprises a Wi-Fi module.

9. The SDV of claim 1 , wherein the first and the second teleassistance inquiries include telemetry data for the SDV.

10. The SDV of claim 1 , wherein the first and the second teleassistance inquiries include localization information for the SDV.

11. The SDV of claim 1 , wherein the first and the second teleassistance inquiries include diagnostics data for the SDV.

12. A non-transitory computer readable medium storing instructions that, when executed by a control system of a self-driving vehicle (SDV), cause the control system to:
analyze a live sensor view of the SDV in accordance with a safety threshold to (i) detect objects of interest along a current route, (ii) classify each of the detected objects of interest in accordance with a certainty threshold, and (iii) autonomously operate acceleration, braking, and steering systems of the SDV along the current route, wherein the live sensor view comprises LIDAR data from one or more LIDAR sensors and image data from one or more cameras of the SDV;
when the safety threshold is not met, transmit a first teleassistance inquiry to a backend transport system, the first teleassistance inquiry comprising LIDAR data;
when the certainty threshold for an indeterminate object is not met, transmit a second teleassistance inquiry to the backend transport system, the second teleassistance inquiry comprising image data that includes the indeterminate object;
for each of the first and the second teleassistance inquiries, receive a resolution response from the backend transport system; and
autonomously operate the acceleration, braking, and steering systems to proceed in accordance with the resolution response.

13. The non-transitory computer readable medium of claim 12 , wherein the safety threshold not being met corresponds to an obstruction in the live sensor view.

14. The non-transitory computer readable medium of claim 12 , wherein the executed instructions cause the control system to transmit LIDAR data and not image data when the safety threshold is not met.

15. The non-transitory computer readable medium of claim 12 , wherein the SDV comprises a plurality of wireless communication modules enabling communication with the backend transport system via a plurality of wireless communication providers, and wherein the executed instructions further cause the control system to:
monitor bandwidth of each of the plurality of wireless communication modules to determine a highest bandwidth communication module;
wherein the executed instructions cause the control system to transmit each of the first and the second teleassistance inquiries to the backend transport system using the highest bandwidth communication module.

16. A computer-implemented method of operating a self-driving vehicle (SDV), the method being performed by a control system of the SDV and comprising:
analyzing a live sensor view of the SDV in accordance with a safety threshold to (i) detect objects of interest along a current route, (ii) classify each of the detected objects of interest in accordance with a certainty threshold, and (iii) autonomously operate acceleration, braking, and steering systems of the SDV along the current route, wherein the live sensor view comprises LIDAR data from one or more LIDAR sensors and image data from one or more cameras of the SDV;
in response to the safety threshold not being met, transmit a first teleassistance inquiry to a backend transport system, the first teleassistance inquiry comprising LIDAR data;
in response to the certainty threshold for an indeterminate object not being met, transmit a second teleassistance inquiry to the backend transport system, the second teleassistance inquiry comprising image data that includes the indeterminate object;
for each of the first and the second teleassistance inquiries, receive a resolution response from the backend transport system; and
autonomously operate the acceleration, braking, and steering systems to proceed in accordance with the resolution response.

Assignees

• Uber Technologies Inc

Inventors

• Kroop Benjamin
• Ross William
• Heine Andrew

Classifications

• B60W10/18

  including control of braking systems · CPC title

• H04W84/12

  WLAN [Wireless Local Area Networks] · CPC title

• G07C5/008

  communicating information to a remotely located station (transmission systems for measured values G08C) · CPC title

• B60W10/04

  including control of propulsion units · CPC title

• B60W2710/18

  Braking system · CPC title