Autonomous vehicle virtual reality navigation system

Patent metadata

Field	Value
Publication number	US-10921138-B2
Application number	US-201815981418-A
Country	US
Kind code	B2
Filing date	May 16, 2018
Priority date	Mar 22, 2016
Publication date	Feb 16, 2021
Grant date	Feb 16, 2021

Abstract

The virtual reality navigation system can display the position of an autonomous vehicle at a predetermined amount of time in the future via virtual reality. A virtual reality display can display a virtual reality autonomous vehicle, as if the virtual reality autonomous vehicle is the autonomous vehicle operating at the predetermined amount of time in the future. Additionally, a virtual reality interface can provide an interface for an operator to interact with the environment of the autonomous vehicle. The virtual reality interface can receive an alternate route selection, and display in virtual reality the one or more maneuvers that would be required to reach the selection.

First claim

The invention claimed is:

1. A device comprising:
circuitry configured to:
receive output from a plurality of sensors,
receive a current position of a vehicle,
display the vehicle via a display at a first predetermined amount of time in the future based on the output from the plurality of sensors and the current position of the vehicle,
receive a selection for a first alternate route,
determine if the first alternate route is available,
display the vehicle executing one or more maneuvers to reach the first alternate route when the first alternate route is available, and
display the vehicle on the first alternate route in a position corresponding to the first predetermined amount of time in the future when the first alternate route is available.

2. The device of claim 1 , wherein the circuitry is further configured to
automatically suggest a second alternate route when the first alternate route is not available, the second alternate route being an available alternate route most similar to the first alternate route,
determine if the second alternate route is selected within a second predetermined amount of time,
display the vehicle executing one or more maneuvers to reach the second alternate route when the second alternate route is selected within the predetermined amount of time, and
display the vehicle on the second alternate route in a position corresponding to the first predetermined amount of time in the future when the second alternate route is selected within the second predetermined amount of time.

3. The device of claim 2 , wherein the first predetermined amount of time is indicative of the position of the vehicle that amount of time in the future, the position of the vehicle being a predetermined position based on a planned route of the autonomous vehicle.

4. The device of claim 3 , wherein the first predetermined amount of time can be adjusted via a virtual reality interface.

5. The device of claim 1 , wherein the display displays a 2 D animation of the vehicle and its surroundings operating at the first predetermined amount of time in the future.

6. The device of claim 1 , wherein the display displays a 3 D animation of the vehicle and its surroundings operating at the first predetermined amount of time in the future.

7. The device of claim 6 , wherein the display is a display system including a Heads Up Display (HUD), a hologram projection system, or a liquid crystal display (LCD).

8. The device of claim 2 , wherein the selection for the first alternate route is an interaction via a virtual reality interface.

9. The device of claim 2 , wherein the automatic suggestion for the second alternate route is displayed on a virtual reality interface.

10. The device of claim 9 , wherein the second alternate route is confirmed within the second predetermined amount of time via an interaction with the virtual reality interface.

11. The device of claim 9 , wherein the second alternate route is automatically suggested when the autonomous vehicle is not able to safely execute one or more maneuvers to reach the first alternate route.

12. The device of claim 2 , wherein the circuitry is configured to not execute a maneuver to any alternate route when the second alternate route is not selected within the predetermined amount of time.

13. The device of claim 2 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed as an animation on the display.

14. The device of claim 2 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed as an animation on a virtual reality interface.

15. The device of claim 2 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed as an animation on the display and a virtual reality interface.

16. The device of claim 15 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed up until the first predetermined amount of time in the future.

17. A method comprising:
receiving output from a plurality of sensors;
receiving a current position of a vehicle;
displaying the vehicle via a display at a first predetermined amount of time in the future based on the output from the plurality of sensors and the current position of the vehicle;
receiving a selection for a first alternate route;
determining, via processing circuitry, if the first alternate route is available;
displaying the vehicle executing one or more maneuvers to reach the first alternate route when the first alternate route is available; and
displaying the vehicle on the first alternate route in a position corresponding to the first predetermined amount of time in the future when the first alternate route is available.

18. The method of claim 17 , further comprising:
automatically suggesting a second alternate route when the first alternate route is not available, the second alternate route being an available alternate route most similar to the first alternate route;
determining, via processing circuitry, if the second alternate route is selected within a second predetermined amount of time;
displaying the vehicle executing one or more maneuvers to reach the second alternate route when the second alternate route is selected within the predetermined amount of time; and
displaying the vehicle on the second alternate route in a position corresponding to the first predetermined amount of time in the future when the second alternate route is selected within the second predetermined amount of time.

19. A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform the method of:
receiving output from a plurality of sensors;
receiving a current position of a vehicle;
displaying the vehicle via a display at a first predetermined amount of time in the future based on the output from the plurality of sensors and the current position of the vehicle;
receiving a selection for a first alternate route;
determining if the first alternate route is available;
displaying the vehicle executing one or more maneuvers to reach the first alternate route when the first alternate route is available; and
displaying the vehicle on the first alternate route in a position corresponding to the first predetermined amount of time in the future when the first alternate route is available.

20. The method of claim 19 , further comprising:
automatically suggesting a second alternate route when the first alternate route is not available, the second alternate route being an available alternate route most similar to the first alternate route;
determining if the second alternate route is selected within a second predetermined amount of time,
displaying the vehicle executing one or more maneuvers to reach the second alternate route when the second alternate route is selected within the predetermined amount of time; and
displaying the vehicle on the second alternate route in a position corresponding to the first predetermined amount of time in the future when the second alternate route is selected within the second predetermined amount of time.

Assignees

• Toyota Eng & Mfg North America

Inventors

• Kusano Kristofer D
• Li Yi
• Hada Hideki
• Johnson Miles J

Classifications

• G01C21/3667

  Display of a road map (G01C21/3614 takes precedence; guidance using 3D or perspective road maps G01C21/3635) · CPC title

• G01C21/365

  Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself · CPC title

• G01C21/3415 Primary

  Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents · CPC title

• B60W50/14

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title