Graphical user interface for display of autonomous vehicle behaviors

Patent metadata

Field	Value
Publication number	US-11472291-B2
Application number	US-202016855954-A
Country	US
Kind code	B2
Filing date	Apr 22, 2020
Priority date	Apr 25, 2019
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

Techniques are disclosed for creating and presenting a graphical user interface for display of autonomous vehicle behaviors. The techniques include determining a trajectory of a vehicle operating in a real-world environment. Sensors of the vehicle obtain sensor data representing an object in the real-world environment. A maneuver of the vehicle to avoid a collision with the object is predicted based on the sensor data and the trajectory of the vehicle. It is determined that a passenger comfort level of a passenger riding in the vehicle will decrease based on the maneuver of the vehicle. The passenger comfort level is measured by passenger sensors of the vehicle. A graphical user interface is generated including representations of the vehicle, the object, and a graphic, text, or a symbol alerting the passenger of the predicted maneuver. The graphical user interface is transmitted to a display device of the vehicle.

First claim

What is claimed is:

1. A method comprising:
measuring, using one or more biometric sensors, passenger comfort level of a passenger riding in a vehicle;
determining, using one or more processors of the vehicle operating in a real-world environment, a trajectory of the vehicle in the real-world environment;
obtaining, using one or more sensors of the vehicle, sensor data representing an object in the real-world environment;
predicting, using the one or more processors, a maneuver of the vehicle to avoid a collision with the object based on the sensor data and the trajectory of the vehicle;
determining that the passenger comfort level will decrease by performing the maneuver;
generating, using the one or more processors, a graphical user interface comprising representations of the vehicle, the object, and a graphic, text, or a symbol alerting a passenger riding in the vehicle of the predicted maneuver, wherein a size or presence of at least one of the representations is based on the passenger comfort level that was measured by at least one of the one or more sensors of the vehicle and the determination that the passenger comfort level will decrease by performing the maneuver; and
transmitting, to a display device in the vehicle, the graphical user interface.

2. The method of claim 1 , further comprising receiving, using an input device of the vehicle, passenger input directed to the object, wherein the object is a dynamic object.

3. The method of claim 2 , further comprising increasing, within the graphical user interface, a size of the object responsive to receiving the passenger input.

4. The method of claim 2 , further comprising increasing, within the graphical user interface, a display resolution of the object responsive to receiving the passenger input.

5. The method of claim 2 , further comprising determining, using the one or more processors, a trajectory of the object in the real-world environment responsive to receiving the passenger input.

6. The method of claim 5 , further comprising displaying the trajectory of the object within the graphical user interface.

7. The method of claim 1 , further comprising obtaining, using the one or more sensors, second sensor data after the obtaining of the sensor data representing the object.

8. The method of claim 7 , wherein the predicting of the maneuver of the vehicle to avoid the collision with the object is performed responsive to the second sensor data matching the sensor data.

9. The method of claim 1 , further comprising determining, using the one or more processors, a level of confidence associated with the generating of the graphical user interface displaying a representation of the object.

10. The method of claim 9 , wherein the transmitting of the graphical user interface is performed responsive to the level of confidence exceeding a threshold value.

11. The method of claim 1 , wherein the generating of the graphical user interface comprises generating, using the one or more processors, a geometric model of the object based on the sensor data.

12. The method of claim 1 , further comprising filtering, using digital signal processing, the sensor data to remove noise from the sensor data.

13. The method of claim 12 , wherein the generating of the graphical user interface is performed based on the filtered sensor data.

14. The method of claim 12 , wherein the predicting of the maneuver of the vehicle to avoid a collision with the object is performed based on the filtered sensor data.

15. The method of claim 1 , wherein the predicting of the maneuver of the vehicle to avoid a collision with the object is performed responsive to the object being located within a threshold distance from the vehicle.

16. The method of claim 1 , wherein the object comprises a second vehicle or a pedestrian.

17. The method of claim 1 , wherein the graphical user interface further displays a three-dimensional (3D) representation of at least one of a lane, a traffic light, a traffic sign, or a construction zone.

18. The method of claim 1 , wherein the generating of the graphical user interface is performed at a first time and the transmitting of the graphical user interface is performed at a second time.

19. A non-transitory computer readable storage medium storing instructions executable by one or more computer processors, the instructions when executed by the one or more computer processors cause the one or more computer processors to:
measuring, using one or more biometric sensors, passenger comfort level of a passenger riding in a vehicle;
determine a trajectory of a vehicle operating in a real-world environment;
obtain, using one or more sensors of the vehicle, sensor data representing an object in the real-world environment;
predict a maneuver of the vehicle to avoid a collision with the object based on the sensor data and the trajectory of the vehicle;
determine that the passenger comfort level will decrease by performing the maneuver;
generate a graphical user interface comprising representations of the vehicle, the object, and a graphic, text, or a symbol alerting a passenger riding in the vehicle of the predicted maneuver, wherein a size or presence of at least one of the representations is based on the passenger comfort level that was measured by at least one of the one or more sensors of the vehicle and the determination that the passenger comfort level will decrease by performing the maneuver; and
transmit, to a display device of in vehicle, the graphical user interface.

20. A vehicle comprising:
one or more computer processors; and
a non-transitory computer readable storage medium storing instructions executable by one or more computer processors, the instructions when executed by the one or more computer processors cause the one or more computer processors to:
measure, using one or more biometric sensors, passenger comfort level of a passenger riding in a vehicle;
determine a trajectory of a vehicle operating in a real-world environment;
obtain, using one or more sensors of the vehicle, sensor data representing an object in the real-world environment;
predict a maneuver of the vehicle to avoid a collision with the object based on the sensor data and the trajectory of the vehicle;
determining that the passenger comfort level will decrease by performing the maneuver;
generate a graphical user interface comprising representations of the vehicle, the object, and a graphic, text, or a symbol alerting a passenger riding in the vehicle of the predicted maneuver, wherein a size or presence of at least one of the representations is based on the passenger comfort level that was measured by at least one of the one or more sensors of the vehicle and the determination that the passenger comfort level will decrease by performing the maneuver; and
transmit, to a display device in the vehicle, the graphical user interface.

21. The method of claim 1 , further comprising:
identifying, using the one or more processors based on at least one measurement by the one or more biometric sensors of the vehicle, the passenger comfort level; and
determining, using the one or more processors based on the passenger comfort level, not to display the representation of the at least one of the vehicle, the object, and the graphic, text, or symbol.

22. The method of claim 1 , further comprising:
identifying, using the one or more processors based on at least one measurement by the one or more biometric sensors of the vehicle, the passenger co

Assignees

• Motional Ad Llc

Inventors

• Alalao Ayman

Classifications

• B60K35/10 Primary

  Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor · CPC title

• B60W2420/00

  Indexing codes relating to the type of sensors based on the principle of their operation · CPC title

• B60W40/10

  related to vehicle motion · CPC title

• G06F9/451 Primary

  Execution arrangements for user interfaces · CPC title

• B60Y2300/08

  Predicting or avoiding probable or impending collision · CPC title