Remote operation of autonomous vehicle in unexpected environment

Patent metadata

Field	Value
Publication number	US-9494935-B2
Application number	US-201414541061-A
Country	US
Kind code	B2
Filing date	Nov 13, 2014
Priority date	Nov 13, 2014
Publication date	Nov 15, 2016
Grant date	Nov 15, 2016

Abstract

Disclosed herein are computer devices, systems, and methods for remotely operating an autonomous passenger vehicle. When an autonomous vehicle encounters an unexpected driving environment unsuitable for autonomous operation, such as road construction or an obstruction, vehicle sensors can capture data about the vehicle and the unexpected driving environment, including images, radar and lidar data, etc. The captured data can be sent to a remote operator. The remote operator can manually operate the vehicle remotely or issue commands to the autonomous vehicle to be executed by on various vehicle systems. The captured data sent to the remote operator can be optimized to conserve bandwidth, such as by sending a limited subset of the captured data.

First claim

What is claimed is:

1. A computing device for an autonomous passenger vehicle comprising:
one or more processors for controlling operations of the computing device;
a communications interface configured to communicate with a remote server over a network; and
a memory for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory to:
identify an unexpected driving environment
send information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote operator using the remote server;
receive a command sent by the remote operator relating to one or more vehicle systems; and
send the command to the one or more vehicle systems for execution, the command being sent based on a quality metric of the network.

2. The computing device of claim 1 , wherein the unexpected driving environment is identified autonomously based at least in part on information received from the one or more sensors disposed on the vehicle.

3. The computing device of claim 1 , wherein the information received from the one or more sensors relates to at least one of: vehicle speed, vehicle direction, vehicle acceleration, vehicle rotation, vehicle location, environmental weather conditions, traffic conditions, road conditions, and images of the unexpected driving environment.

4. The computing device of claim 1 , wherein the vehicle systems include at least one of a steering system, a propulsion system, a stability control system, a navigation system, and an energy system.

5. The computing device of claim 1 , wherein the command received from the remote operator is used to directly control the one or more vehicle systems.

6. The computing device of claim 1 , wherein the vehicle systems are controlled autonomously based on the command received from the remote operator.

7. The computing device of claim 1 , wherein the information received from the one or more sensors is sent to the remote operator upon receiving an indication of approval by an occupant of the vehicle.

8. The computing device of claim 1 , wherein the information received from the one or more sensors is sent to the remote operator upon identification of the unexpected driving environment.

9. The computing device of claim 1 , wherein the information received from the one or more sensors is selectively sent to the remote operator based on the remote operator's need for the information in remotely operating the vehicle.

10. The computing device of claim 1 , wherein additional information relating to the unexpected driving environment is received from other vehicles located at the unexpected driving environment.

11. The computing device of claim 1 , wherein the command is sent to other vehicles located at the unexpected driving environment for autonomously controlling the other vehicles.

12. The computing device of claim 1 , wherein the network is selected based on quality of service.

13. The computing device of claim 1 , wherein the information sent over the network is based at least in part on a quality metric of the network.

14. A computer-implemented method for an autonomous passenger vehicle comprising:
identifying, using one or more processors, an unexpected driving environment if at least
sending information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote operator using a remote server;
receiving a command sent by the remote operator relating to one or more vehicle systems; and
sending the command to the one or more vehicle systems for execution, the command being sent based on a quality metric of the network.

15. The method of claim 14 , wherein the unexpected driving environment is identified autonomously based at least in part on information received from the one or more sensors disposed on the vehicle.

16. The method of claim 14 , wherein the command received from the remote operator is used to directly control the one or more vehicle systems.

17. The method of claim 14 , wherein the vehicle systems are controlled autonomously based on the command received from the remote operator.

18. The method of claim 14 , wherein the information received from the one or more sensors is selectively sent to the remote operator based on the remote operator's need for the information in remotely operating the vehicle.

19. A system comprising:
one or more sensors disposed on a vehicle; and
a computing device in communication with the one or more sensors, comprising:
one or more processors for controlling operations of the computing device;
a communications interface configured to communicate with a remote server over a network; and
a memory for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory to:
identify an unexpected driving environment
send information received from one or more sensors associated with the vehicle to a remote operator using the remote server;
receive a command sent by the remote operator relating to one or more vehicle systems;
send the command to the one or more vehicle systems based on a quality metric of the network; and
execute the command on the one or more vehicle systems.

Assignees

• Toyota Motor Eng & Mfg North America Inc

Inventors

• Okumura Bunyo
• Prokhorov Danil V

Classifications

• B60W30/00

  Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units · CPC title

• G05D1/0022 Primary

  characterised by the communication link (data switching networks in general H04L12/00) · CPC title

• G05D2201/0213

  Physics · mapped topic