System and method for providing situational awareness interfaces for autonomous vehicle operators

Patent metadata

Field	Value
Publication number	US-11955010-B2
Application number	US-202117445079-A
Country	US
Kind code	B2
Filing date	Aug 13, 2021
Priority date	Aug 13, 2021
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

A supervisory control system is disclosed that provides an operator situational awareness interface use with monitoring a plurality of automated vehicles (AVs). The system is configured to: generate a map of a geographical area of interest; obtain location data and perceived risk data for a plurality of AVs in the geographical area; generate a vehicle icon corresponding to each AV; position the vehicle icon for each AV on the map based on the location data for a corresponding AV; apply a color coding to each vehicle icon based on a perceived risk level for a corresponding AV; and signal a display device to display an AV fleet map graphic that includes the color coded vehicle icons positioned on the map. The controller may be further configured to: generate an AV servicing queue graphic that displays vehicle icons in an order based on a determined servicing priority.

First claim

What is claimed is:

1. A supervisory control system for providing an operator situational awareness interface for a plurality of operators, the system comprising a controller configured to:
generate a map of a geographical area of interest;
obtain location data and perceived risk data for a plurality of automated vehicles (AVs) in the geographical area;
generate a vehicle icon corresponding to each of the plurality of AVs;
position the vehicle icon for each of the plurality of AVs on the map based on the location data for the corresponding AV;
apply a color coding to each vehicle icon based on a perceived risk level for the corresponding AV;
generate an AV servicing queue graphic that displays vehicle icons for the plurality of AVs in an order based on a determined servicing priority; and
signal a display device to display an AV fleet map graphic that includes the map with color coded vehicle icons positioned thereon and the AV servicing queue graphic;
wherein to generate the AV servicing queue graphic, the controller is further configured to:
determine the perceived risk level for each of the plurality of AVs based on the perceived risk data;
identify, based on the perceived risk levels, one or more AVs in need of servicing assistance from the plurality of AVs;
determine a servicing assistance prioritization order for the one or more AVs in need of servicing assistance based on the perceived risk levels of the one or more AVs in need of servicing assistance;
generate a second vehicle icon corresponding to each of the one or more AVs in need of servicing assistance;
apply a second color coding to each second vehicle icon for the one or more AVs in need of servicing assistance based on a perceived risk for the corresponding AV; and
position the second vehicle icon for the one or more AVs in need of servicing assistance on the AV servicing queue graphic in the determined servicing assistance prioritization order.

2. The system of claim 1 , wherein to obtain location data and perceived risk data, the controller is configured to retrieve the location data and perceived risk data from a data integration module, wherein the data integration module is configured to accumulate the location data and perceived risk data from the plurality of AVs.

3. The system of claim 1 , wherein the controller is further configured to assign a risk-level-specific color to each vehicle icon based on the perceived risk data for the corresponding AV.

4. The system of claim 1 , wherein the controller is further configured to determine the perceived risk level for each of the plurality of AVs based on the perceived risk data.

5. The system of claim 1 , wherein the controller is further configured to assign one of a plurality of predetermined perceived risk levels to each of the plurality of AVs based on the perceived risk data.

6. The system of claim 5 , wherein the controller is further configured to assign a risk-level-specific color to each vehicle icon based on the perceived risk level assigned to the corresponding AV.

7. The system of claim 1 , wherein the controller is further configured to:
generate and position a prioritization order graphical element that indicates a servicing priority order on the AV servicing queue graphic; and
generate and position, on the AV servicing queue graphic, a plurality of operator icons that are representative of operators who can assist with servicing the needs of the one or more AVs in need of servicing assistance.

8. The system of claim 7 , wherein the controller is further configured to:
generate and position on the AV servicing queue graphic an operator assignment graphical element for use in identifying which of the one or more AVs in need of servicing assistance has been assigned to a particular operator; and
indicate the assignment of an AV to the particular operator by positioning the vehicle icon for the assigned AV on the operator assignment graphical element for the particular operator.

9. The system of claim 7 , wherein the controller is further configured to:
generate and position on the AV servicing queue graphic a special servicing need graphical element for use in identifying AVs that have a special servicing need and to identify an operator who may be best suited to service the special servicing need;
generate and position on the AV servicing queue graphic a special operator assignment graphical element for use in identifying which of the AVs with a special servicing need has been assigned to the operator who is best suited to service the special servicing need; and
indicate the assignment of an AV with a special servicing need to the operator who is best suited to service the special servicing need by positioning the vehicle icon for the assigned special servicing need AV on the special operator assignment graphical element.

10. A method in a supervisory control system for providing an operator situational awareness interface for a plurality of operators, the method comprising:
generating a map of a geographical area of interest;
obtaining location data and perceived risk data for a plurality of automated vehicles (AVs) in the geographical area;
generating a vehicle icon corresponding to each of the plurality of AVs;
positioning the vehicle icon for each of the plurality of AVs on the map based on the location data for the corresponding AV;
applying a color coding to each vehicle icon based on a perceived risk level for the corresponding AV;
generating an AV servicing queue graphic that displays vehicle icons for the plurality of AVs in an order based on a determined servicing priority; and
signaling a display device to display an AV fleet map graphic that includes the map with color coded vehicle icons positioned thereon and the AV servicing queue graphic
wherein generating the AV servicing queue graphic comprises:
determining the perceived risk level for each of the plurality of AVs based on the perceived risk data;
identifying, based on the perceived risk levels, one or more AVs in need of servicing assistance;
determining a servicing assistance prioritization order for the one or more AVs in need of servicing assistance based on the perceived risk levels of the one or more AVs in need of servicing assistance;
generating a second vehicle icon corresponding to each of the one or more AVs in need of servicing assistance;
applying a second color coding to each second vehicle icon for the one or more AVs in need of servicing assistance based on a perceived risk for the corresponding AV; and
positioning the second vehicle icon for the one or more AVs in need of servicing assistance on the AV servicing queue graphic in the determined servicing assistance prioritization order.

11. The method of claim 10 , further comprising assigning one of a plurality of predetermined perceived risk levels to each of the plurality of AVs based on the perceived risk data.

12. The method of claim 11 , further comprising assigning a risk-level-specific color to each vehicle icon based on the perceived risk level assigned to the corresponding AV.

13. The method of claim 10 , further comprising:
generating and positioning a prioritization order graphical element that indicates a servicing priority order on the AV servicing queue graphic; and
generating and positioning, on the AV servicing queue graphic, a plurality of operator icons that are representative of operators who can assist with servicing the needs of the one or more AVs in need of servicing assistance.

14. The method of claim 13 , further comprising:
generating and positioning on the AV servicing queue graphic an operator

Assignees

• Gm Global Tech Operations Llc

Inventors

• Bush Lawrence A
• Mathieu Roy J
• Szczerba Joseph F

Classifications

• G08G1/20 Primary

  Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles · CPC title

• G05D1/0027

  involving a plurality of vehicles, e.g. fleet or convoy travelling (fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00) · CPC title

• G05D1/0044

  by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005) · CPC title

• G06F3/04817

  using icons (graphical or visual programming using iconic symbols G06F8/34) · CPC title

• G07C5/006

  Indicating maintenance · CPC title