Interactive autonomous vehicle command controller

Patent metadata

Field	Value
Publication number	US-10248119-B2
Application number	US-201514933706-A
Country	US
Kind code	B2
Filing date	Nov 5, 2015
Priority date	Nov 4, 2015
Publication date	Apr 2, 2019
Grant date	Apr 2, 2019

Abstract

Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. In particular, a method may include receiving, at an autonomous vehicle system, a command to control an ambient feature associated with the autonomous vehicle system. One or more courses of action may be determined based on the command. In addition, one or more probabilistic models associated with the one or more courses of action may also be determined. Based on the one or more probabilistic models, confidence levels may also be determined to form a subset of the one or more courses of action. A course of action from the subset of the one or more courses of action may then be executed at the autonomous vehicle system responsive to the command.

First claim

What is claimed:

1. A method comprising:
receiving, from a sensor disposed in an autonomous vehicle, a signal corresponding to an audible request to control a route of the autonomous vehicle;
extracting one or more words from the signal;
determining one or more generative probabilistic models for interpreting the one or more words;
determining, based at least in part on the one or more generative probabilistic models, a probability associated with a first word of the one or more words;
determining that the probability is equal to or below a threshold value;
sending, based at least in part on the probability being equal to or below the threshold value, the signal to a teleoperator system;
receiving, from the teleoperator system, a command to control the route of the autonomous vehicle system in accordance with the request;
determining a plurality of courses of action based on the command;
for each of the plurality of courses of action, determining at least one second probabilistic model;
based on the second probabilistic model, determining a confidence level for each of the plurality of courses of action;
forming a subset of the plurality of courses of action, each course of action in the subset having a confidence level of at least a threshold confidence level; and
executing a course of action from the subset at the autonomous vehicle system.

2. The method of claim 1 , wherein the sensor is a microphone mounted in the autonomous vehicle system.

3. The method of claim 2 , further comprising:
responsive to the command, generating an audio response to the command based on the one or more probabilistic models; and
providing the response through one or more interior speakers mounted within the autonomous vehicle system.

4. A method comprising:
receiving, from a first sensor of an autonomous vehicle system comprising a plurality of sensors, a signal corresponding to an audible request to change a route associated with the autonomous vehicle system;
determining one or more generative probabilistic models for interpreting at least one word in the signal;
determining, based at least in part on the one or more generative probabilistic models, a probability associated with a first word of the at least one word;
sending, based at least in part on the probability being equal to or below a threshold value, the signal to a teleoperator system;
receiving, from the teleoperator system, a command to alter the route of the autonomous vehicle system;
determining courses of action based on the command and data received from the plurality of sensors;
determining, for each of the courses of action, a second probabilistic model;
determining a confidence level for each of the courses of action, based at least in part on the second probabilistic model; and
executing, based at least in part on the confidence levels, a course of action from the courses of action at the autonomous vehicle system responsive to the command.

5. The method of claim 4 , wherein the course of action comprises an instruction to contact the teleoperator system to respond to the command, the method further comprising:
sending the command to the teleoperator system;
receiving a second course of action from the teleoperator system; and
executing the second course of action.

6. The method of claim 4 , further comprising selecting the course of action from the one or more courses of action based at least in part on the confidence levels.

7. The method of claim 6 , wherein the course of action has an associated confidence level higher than a predetermined threshold level.

8. The method of claim 6 , wherein the selecting the course of action comprises selecting the course of action from a subset of the one or more courses of action, each of the courses of action in the subset having a confidence level higher than a predetermined threshold level.

9. The method of claim 4 , wherein the confidence levels are determined based at least in part on at least one of data from the sensors, log data, or information provided by an autonomous vehicle management system.

10. A system comprising:
an autonomous vehicle configured to drive autonomously on a roadway, the autonomous vehicle including one or more sensors for sensing one or more objects in an environment of the autonomous vehicle; and
a computing system communicatively coupled to the autonomous vehicle to receive data and to transmit instructions to control the autonomous vehicle, the computing system being programmed to:
control the autonomous vehicle to transport a user to a first destination along a trajectory;
receive a signal corresponding to an audible request from a user to transport the user to a second destination;
extract one or more words from the signal;
determine one or more generative probabilistic models for interpreting the one or more words;
determine, based at least in part on the one or more generative probabilistic models, a probability associated with a first word of the one or more words;
determine that the probability is equal to or below a threshold value;
sending, based at least in part on the probability being equal to or below the threshold value, the one or more words to a teleoperator system;
receive, from the teleoperator system, a command to alter the route of the autonomous vehicle system;
determine a plurality of courses of action in response to the command;
determine a confidence level for each of the courses of action; and
control the autonomous vehicle to execute, based at least in part on the confidence levels, a course of action of the courses of action.

11. The system of claim 10 , the computing system being further programmed to: determine a probabilistic model for each of the courses of action, wherein the confidence level is determined based at least in part on the probabilistic models.

12. The system of claim 10 , wherein the confidence level of the course of action is higher than a predetermined threshold level.

13. The system of claim 10 , wherein the confidence levels are determined based at least in part on data from the sensors.

14. The method of claim 1 , wherein the interpreting the one or more words comprises an initial guess that converges toward an optimal response through a series of step-wise increments, using the one or more generative probabilistic models.

15. The method of claim 1 , further comprising:
based at least in part on the probability, determining a response to the audible request using the one or more generative probabilistic models; and
outputting the response to a passenger of the autonomous vehicle.

16. The method of claim 4 , wherein the determining the probability comprises an initial guess that converges toward an optimal response through a series of step-wise increments, using the one or more generative probabilistic models.

17. The method of claim 4 , further comprising:
sending the signal to a teleoperator system, the teleoperator system configured to generate the at least one word based, at least in part, on the signal and generate the command.

18. The system of claim 10 , wherein the computing system is programmed to determine the probability by generating an initial guess that converges toward an optimal response through a series of step-wise increments, using the one or more generative probabilistic models.

19. The system of claim 10 , wherein the computing system is further programmed to:
determine a response to the audible request using the one or more generative probabilistic models; and
outputting the res

Assignees

• Kentley Klay Timothy David
• Gamara Rachad Youssef
• Zoox Inc

Inventors

• Kentley-Klay Timothy David
• Gamara Rachad Youssef

Classifications

• B60Q1/28

  for indicating front of vehicle · CPC title

• B60L15/20

  for control of the vehicle or its driving motor to achieve a desired performance, e.g. speed, torque, programmed variation of speed · CPC title

• B60L2260/32

  Auto pilot mode · CPC title

• Y02T10/72

  Electric energy management in electromobility · CPC title

• Y02T90/16

  Information or communication technologies improving the operation of electric vehicles · CPC title