Systems and methods for a parallel autonomy interface

Patent metadata

Field	Value
Publication number	US-10311726-B2
Application number	US-201715656422-A
Country	US
Kind code	B2
Filing date	Jul 21, 2017
Priority date	Jul 21, 2017
Publication date	Jun 4, 2019
Grant date	Jun 4, 2019

Abstract

A parallel autonomy interface system can include processing circuitry communicably coupling one or more sensors of an autonomous vehicle and one or more displays of the autonomous vehicle. The processing circuitry can be configured to receive output from the one or more sensors, determine if potential danger with respect to vehicle operation is detected based on the output of the one or more sensors, activate an alert in response to detecting the potential danger, the alert corresponding to avatar motion, and enable the avatar to indicate a direction of the potential danger in response to activation of the alert.

First claim

The invention claimed is:

1. A parallel autonomy interface system, comprising:
one or more sensors coupled to a vehicle capable of autonomous operation;
one or more displays configured to display an avatar, the avatar including at least a head and a hand; and
processing circuitry communicably coupling the one or more sensors and the one or more displays, the processing circuitry configured to
receive output from the one or more sensors,
determine if potential danger with respect to vehicle operation is detected based on the output of the one or more sensors,
activate an alert in response to detecting the potential danger, the alert corresponding to a change in a size of the avatar, and
cause the avatar to indicate a direction of the potential danger, in response to activation of the alert, by turning the head of the avatar to face the potential danger and pointing hand of the avatar towards the potential danger.

2. The parallel autonomy interface system of claim 1 , wherein the processing circuitry is configured to
display the avatar in a first location, the first location being within a predetermined area relative to a first focal point of the operator of the vehicle,
determine if the first focal point changes to a second focal point of the operator of the vehicle,
determine if the avatar is in the predetermined area relative to the first focal point in response to the first focal point changing to the second focal point, and
display the avatar in a second location in response to the avatar not being in the predetermined area relative to the second focal point, the second location being within the predetermined area relative to the second focal point.

3. The parallel autonomy interface system of claim 2 , wherein the predetermined area corresponds to a predetermined range of peripheral vision of the operator of the vehicle.

4. The parallel autonomy interface system of claim 1 , wherein the processing circuitry is configured to
determine if the one or more displays are producing sound, and
reduce a volume of the sound produced from the one or more displays in response to the potential danger being detected and the one or more displays producing sound.

5. The parallel autonomy interface system of claim 2 , wherein the first location corresponds to a first display and the second location corresponds to a second display.

6. The parallel autonomy interface system of claim 1 , wherein the avatar can be selected by the operator from a plurality of avatars.

7. The parallel autonomy interface system of claim 1 , wherein the avatar further indicates the direction of the potential danger by walking in the direction of the potential danger.

8. A method of operating a parallel autonomy interface, comprising:
receiving output from one or more sensors, the one or more sensors communicably coupled to one or more displays;
determining, via processing circuitry, if potential danger with respect to vehicle operation is detected based on the output of the one or more sensors;
activating an alert in response to detecting the potential danger, the alert corresponding to a change in a size of a displayed avatar, the avatar including at least a head and a hand; and
causing the avatar to indicate a direction of the potential danger, in response to activation of the alert, by turning the head of the avatar to face the potential danger and pointing the hand of the avatar towards the potential danger.

9. The method of claim 8 , further comprising:
displaying the avatar, via the one or more displays, in a first location, the first location being within a predetermined area relative to a first focal point of an operator of the vehicle;
determining if the first focal point changes to a second focal point of the operator of the vehicle;
determining if the avatar is in the predetermined area relative to the first focal point in response to the first focal point changing to the second focal point; and
displaying the avatar, via the one or more displays, in a second location in response to the avatar not being in the predetermined area relative to the second focal point, the second location being within the predetermined area relative to the second focal point.

10. The method of claim 9 , wherein the predetermined area corresponds to a predetermined range of peripheral vision of the operator of the vehicle.

11. The method of claim 8 , further comprising:
determining if the one or more displays are producing sound; and
reducing a volume of the sound produced from the one or more displays in response to the potential danger being detected and the one or more displays producing sound.

12. The method of claim 9 , wherein the first location corresponds to a first display and the second location corresponds to a second display.

13. The method of claim 8 , wherein the avatar can be selected by an operator of the vehicle from a plurality of avatars.

14. The method of claim 8 , wherein the avatar further indicates the direction of the potential danger by walking in the direction of the potential danger.

15. A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method of operating a parallel autonomy interface, comprising:
receiving output from one or more sensors, the one or more sensors communicably coupled to one or more displays;
determining if potential danger with respect to vehicle operation is detected based on the output of the one or more sensors;
activating an alert in response to detecting the potential danger, the alert corresponding to a change in a size of a displayed avatar, the avatar including at least a head and a hand; and
causing the avatar to indicate a direction of the potential danger, in response to activation of the alert, by turning the head of the avatar to face the potential danger and pointing the hand of the avatar towards the potential danger.

16. The method of claim 15 , further comprising:
displaying the avatar, via the one or more displays, in a first location, the first location being within a predetermined area relative to a first focal point of an operator of the vehicle;
determining if the first focal point changes to a second focal point of the operator of the vehicle;
determining if the avatar is in the predetermined area relative to the first focal point in response to the first focal point changing to the second focal point; and
displaying the avatar, via the one or more displays, in a second location in response to the avatar not being in the predetermined area relative to the second focal point, the second location being within the predetermined area relative to the second focal point.

17. The method of claim 15 , further comprising:
determining if the one or more displays are producing sound; and

reducing a volume of the sound produced from the one or more displays in response to the potential danger being detected and the one or more displays producing sound.

Assignees

• Toyota Res Inst Inc

Inventors

• Patrikalakis Andrew
• Michel Philipp

Classifications

• G06F3/012

  Head tracking input arrangements · CPC title

• G08G1/166

  for active traffic, e.g. moving vehicles, pedestrians, bikes · CPC title

• G09G2380/10

  Automotive applications · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• B60R2300/8093

  for obstacle warning · CPC title