Providing remote assistance to an autonomous vehicle

Patent metadata

Field	Value
Publication number	US-10345809-B2
Application number	US-201514711570-A
Country	US
Kind code	B2
Filing date	May 13, 2015
Priority date	May 13, 2015
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

An event is detected that impairs a confidence level of the autonomous vehicle in progressing through a current route. In response to detecting the event, the autonomous vehicle communicates information about the event to a remote source of guidance. The autonomous vehicle can receive instructions from the remote source of guidance on how to handle the event. The autonomous vehicle can then implement the instructions to handle the event while it operates.

First claim

What is claimed is:

1. A method for remotely guiding an autonomous vehicle, the method being implemented by one or more processors and comprising:
processing an alert from the autonomous vehicle that includes sensor information, the alert indicating that the autonomous vehicle is uncertain in traversing a segment of roadway while on a trip;
displaying an interface for a human operator who is remote from the autonomous vehicle, the interface including a visual representation of the autonomous vehicle and a set of interactive features that is generated from the sensor information;
detecting a first selection input from the remote human operator on a selected region of the interface;
identifying a specific set of sensors based on the selected region of the interface and the visual representation of the autonomous vehicle;
converting the first selection input into a request for additional sensor data from the specific set of sensors of the autonomous vehicle;
communicating the request to the autonomous vehicle, the request causing the autonomous vehicle to obtain the additional sensor data collected from the specific set of sensors;
receiving the additional sensor data from the autonomous vehicle;
displaying the additional sensor data on the interface;
detecting a second selection input from the remote human operator of one or more of the interactive features in the set of interactive features, the second selection input specifying a response strategy comprising a condition and a plurality of actions for the autonomous vehicle to perform without additional input from the remote human operator; and
communicating the response strategy to the autonomous vehicle, the response strategy causing the autonomous vehicle to perform at least some of the plurality of actions, to monitor for the condition while approaching or traversing the segment of roadway, and in response to detecting the condition, to perform one or more actions to change a direction, speed, or driving lane of the autonomous vehicle without additional input from the remote human operator.

2. The method of claim 1 , wherein displaying the interface includes superimposing one or more interactive features of the interface over content that is generated from the sensor information.

3. The method of claim 1 , wherein displaying the interface includes displaying a time remaining for the remote human operator to provide the second selection input before the autonomous vehicle performs a default action.

4. A non-transitory computer-readable medium that stores instructions for remotely guiding an autonomous vehicle, the instructions, when executed by one or more processors, causing the one or more processors to perform operations comprising:
processing an alert from the autonomous vehicle that includes sensor information, the alert indicating that the autonomous vehicle is uncertain in traversing a segment of roadway while on a trip;
displaying an interface for a human operator who is remote from the autonomous vehicle, the interface including a visual representation of the autonomous vehicle and a set of interactive features that is generated from the sensor information;
detecting a first selection input from the remote human operator on a selected region of the interface;
identifying a specific set of sensors based on the selected region of the interface and the visual representation of the autonomous vehicle;
converting the first selection input into a request for additional sensor data from the specific set of sensors of the autonomous vehicle;
communicating the request to the autonomous vehicle, the request causing the autonomous vehicle to obtain the additional sensor data collected from the specific set of sensors;
receiving the additional sensor data from the autonomous vehicle;
displaying the additional sensor data on the interface;
detecting a second selection input from the remote human operator of one or more of the interactive features in the set of interactive features, the second selection input specifying a response strategy comprising a condition and a plurality of actions for the autonomous vehicle to perform without additional input from the remote human operator; and
communicating the response strategy to the autonomous vehicle, the response strategy causing the autonomous vehicle to perform at least some of the plurality of actions, to monitor for the condition while approaching or traversing the segment of roadway, and in response to detecting the condition, to perform one or more actions to change a direction, speed, or driving lane of the autonomous vehicle without additional input from the remote human operator.

5. The method of claim 1 , wherein displaying the interface includes displaying image content captured by one or more cameras of the autonomous vehicle as the autonomous vehicle approaches the segment of roadway.

6. The method of claim 1 , wherein displaying the interface includes providing an interactive feature to enable the remote human operator to request additional information from the autonomous vehicle.

7. The method of claim 6 , wherein the interactive feature enables the remote human operator to request additional images from any one of multiple cameras on the autonomous vehicle.

8. The method of claim 6 , wherein the interactive feature enables the remote human operator to directionally control one or more sensors of the autonomous vehicle to obtain additional information from the autonomous vehicle.

9. The method of claim 1 , wherein displaying the interface includes providing an interactive feature to enable the remote human operator to communicate an instruction to the autonomous vehicle to take no action to avoid a detected object or event that is causing uncertainty for the autonomous vehicle.

10. The method of claim 1 , wherein displaying the interface includes providing multiple interactive features to enable the remote human operator to communicate one or more instructions that correspond to a steering or braking operation that is to be performed by the autonomous vehicle.

11. The method of claim 10 , wherein the multiple interactive features enable the remote human operator to communicate one or more instructions that specify a magnitude of the steering or braking operation.

12. The method of claim 1 , wherein displaying the interface for the remote human operator includes providing one or more features that enable the remote human operator to view images that are above the autonomous vehicle or next to the segment of roadway.

13. The non-transitory computer readable medium of claim 4 , wherein one of the interactive features in the set of interactive features is to enable the remote human operator to instruct a null direction or speed adjustment.

14. The non-transitory computer readable medium of claim 4 , wherein displaying the interface includes superimposing one or more interactive features of the interface over content that is generated from the sensor information.

15. The non-transitory computer readable medium of claim 4 , wherein displaying the interface includes displaying a time remaining for the remote human operator to provide the second selection input before the autonomous vehicle performs a default action.

16. The non-transitory computer readable medium of claim 4 , wherein displaying the interface includes displaying image content captured by one or more cameras of the autonomous vehicle as the autonomous vehicle approaches the segment of roadway.

17. The non-transitory computer readable medium of claim 4 , wherein displaying the interface includes pro

Assignees

• Uber Technologies Inc

Inventors

• Ross William
• Bares John
• Larose David
• Sweeney Matthew

Classifications

• G01C21/3415 Primary

  Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents · CPC title

• G05D1/0088 Primary

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• B60W30/165 Primary

  Automatically following the path of a preceding lead vehicle, e.g. "electronic tow-bar" · CPC title

• B60W60/0059

  Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity · CPC title

• G05D1/0027

  involving a plurality of vehicles, e.g. fleet or convoy travelling (fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00) · CPC title