Solution path overlay interfaces for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-10890915-B2
Application number	US-201716463236-A
Country	US
Kind code	B2
Filing date	Dec 6, 2017
Priority date	Dec 6, 2016
Publication date	Jan 12, 2021
Grant date	Jan 12, 2021

Abstract

Methods and systems for generating a solution path overlay interface to transmit a solution path are described. The disclosed technology includes receiving vehicle data and external data from a vehicle. The vehicle data includes a vehicle location and a vehicle destination, and the external data includes a location and a movement path for each of a plurality of external objects. A solution path is determined between the vehicle location and the vehicle destination, wherein the solution path does not intersect with the plurality of external objects. A solution path overlay interface is generated that includes the vehicle traveling the solution path and at least some of the plurality of external objects. The solution path overlay interface is outputted for display that is configured to receive a command from an operator which results in an updated solution path that is transmitted to the vehicle for execution.

First claim

What is claimed is:

1. A method, comprising:
receiving, by a solution path overlay system, vehicle data and external data from a vehicle control system of a vehicle, wherein the vehicle data is associated with the vehicle and includes a vehicle location and a vehicle destination, and the external data includes a location and a movement path for each of a plurality of external objects;
determining, by the solution path overlay system, a solution path between the vehicle location and the vehicle destination using the vehicle data and the external data;
generating, by the solution path overlay system, a solution path overlay interface that includes an indicator of the solution path, an indicator of a solution path overlay that corresponds to the solution path, an indicator of a roadway, a representation of the vehicle traversing along the solution path via the roadway, and an indicator for each of at least some of the plurality of external objects, the solution path overlay representing outer edges of the solution path based on a width of the vehicle;
outputting, by the solution path overlay system, the solution path overlay interface for display to an operator of the solution path overlay interface;
in response to detecting a change associated with the movement path of any of the plurality of external objects, receiving, by the solution path overlay system, a command from the operator of the solution path overlay interface, the command including a drawing gestured directly onto the solution path overlay interface and an indication of one or more stop points;
updating, by the solution path overlay system, the solution path overlay interface based on the received command to provide an updated solution path and an updated solution path overlay that corresponds to the updated solution path; and
transmitting, by the solution path overlay system, the updated solution path and the updated solution path overlay to the vehicle for execution to navigate to the vehicle destination.

2. The method of claim 1 , wherein any of the vehicle data and the external data includes sensor data from any of an optical sensor, a light detection and ranging sensor (LIDAR), a motion sensor, and an audio sensor.

3. The method of claim 2 , further comprising:
determining, by the solution path overlay system, a surface state within a predetermined distance of the vehicle based on the sensor data; and
generating, by the solution path overlay system, a surface image including a surface state indication based on the surface state, wherein the solution path overlay interface includes the surface state indication.

4. The method of claim 2 , further comprising:
determining, by the solution path overlay system, traffic conditions using the sensor data and movement paths of the plurality of external objects within a predetermined area, wherein a vehicle route of the vehicle is determined using the traffic conditions, wherein the vehicle route provides navigation data for the vehicle between the vehicle location and the vehicle destination.

5. The method of claim 4 , further comprising:
determining, by a solution path overlay system, a direction of regulated traffic flow based on any of traffic flow data and traffic flow indicators within the predetermined area, wherein the traffic flow data includes authorized traffic flow patterns for the plurality of external objects, wherein the solution path overlay interface includes an indication of the direction of regulated traffic flow.

6. The method of claim 1 , wherein the vehicle location is based on map data that includes a geographical area of the vehicle and of the plurality of external objects, further comprising:
displaying the updated solution path to an operator of the vehicle.

7. The method of claim 1 , further comprising:
determining, by the solution path overlay system, identification data of the plurality of external objects based on any of the vehicle data and the external data;
determining, by the solution path overlay system, mobility states of the plurality of external objects, wherein the mobility states include moveable states and immoveable states; and
updating, by the solution path overlay system and based on the mobility states, the solution path overlay interface to indicate the moveable states and the immoveable states of the plurality of external objects.

8. The method of claim 7 , wherein the identification data comprises any of pedestrians, vehicles, traffic light signals, traffic signage, pavement markings, buildings, primary roadways, and secondary roadways.

9. The method of claim 7 , further comprising:
determining, by the solution path overlay system for each portion of the solution path, a buffer distance between the vehicle and each of the plurality of external objects, wherein the determining of the solution path further comprises dynamically maintaining the buffer distance between the vehicle and each of the plurality of external objects.

10. A solution path overlay system comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
receive vehicle data and external data from a vehicle control system of a vehicle, wherein the vehicle data is associated with the vehicle and includes a vehicle location and a vehicle destination, and the external data includes a location and a movement path for each of a plurality of external objects;
determine a solution path between the vehicle location and the vehicle destination using the vehicle data and the external data;
generate a solution path overlay interface that includes an indicator of the solution path, an indicator of a solution path overlay that corresponds to the solution path, an indicator of a roadway, a representation of the vehicle traversing along the solution path via the roadway, and an indicator for each of at least some of the plurality of external objects, the solution path overlay representing outer edges of the solution path based on a width of the vehicle;
output the solution path overlay interface for display to an operator of the solution path overlay interface;
in response to detecting a change associated with the movement path of any of the plurality of external objects, receive a command from the operator of the solution path overlay interface, the command including a drawing gestured directly onto the solution path overlay interface and an indication of one or more stop points;
update the solution path overlay interface based on the received command to provide an updated solution path and an updated solution path overlay that corresponds to the updated solution path; and
transmit the updated solution path and the updated solution path overlay to the vehicle for execution to navigate to the vehicle destination.

11. The solution path overlay system of claim 10 , wherein any of the vehicle data and the external data includes sensor data from any of an optical sensor, a light detection and ranging sensor (LIDAR), a motion sensor, and an audio sensor.

12. The solution path overlay system of claim 11 , wherein the processor is further configured to execute instructions stored in the memory to:
determine a surface state within a predetermined distance of the vehicle based on the sensor data; and
generate a surface image including a surface state indication based on the surface state, wherein the solution path overlay interface includes the surface state indication.

13. The solution path overlay system of claim 11 , wherein the processor is further configured to execute instructions stored in the memory to:
determine traffic conditions using the sensor data and movement paths of the pluralit

Assignees

• Nissan North America Inc
• Nasa

Inventors

• Pedersen Liam
• Sierhuis Maarten
• Utz Hans
• Penna Mauro Della
• Fong Terrence
• Allan Mark
• Bualat Maria
• Schafer Eric

Classifications

• G01C21/3407 Primary

  specially adapted for specific applications · CPC title

• G01C21/36

  Input/output arrangements for on-board computers · CPC title

• G01C21/3676

  Overview of the route on the road map · CPC title

• G05D2201/0213

  Physics · mapped topic

• G05D1/0044

  by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005) · CPC title