Vehicle operator awareness system

Patent metadata

Field	Value
Publication number	US-12269494-B2
Application number	US-202017121199-A
Country	US
Kind code	B2
Filing date	Dec 14, 2020
Priority date	Aug 31, 2018
Publication date	Apr 8, 2025
Grant date	Apr 8, 2025

Abstract

Systems and methods for maintaining autonomous vehicle operator awareness are provided. A method can include determining, by a computing system, an awareness challenge for an operator of a vehicle. The awareness challenge can be based on object data. The awareness challenge can have one or more criteria. The criteria can include a challenge response interval, a response time, and an action for satisfying the awareness challenge. The method can include initiating, by the computing system, a timer measuring elapsed time from a start time of the challenge response interval. The method can include communicating to the operator, by the computing system, a soft notification indicative of the awareness challenge during the challenge response interval. The method can include determining, by the computing system, whether the operator provides a user input after the response time interval and whether the user input corresponds to the action for satisfying the awareness challenge.

First claim

What is claimed is:

1. A computer-implemented method of generating an awareness challenge for a vehicle operator, comprising:
providing, by a computing system comprising one or more computing devices, a prompt of an awareness challenge to an operator of a vehicle;
initiating, by the computing system, a timer measuring an elapsed time from a start time of a challenge response interval;
communicating, by the computing system, to the operator of the vehicle a soft notification indicative of the awareness challenge during the challenge response interval, wherein the soft notification includes a changing visual stimulus implemented by a series of lights that changes in proportion to the elapsed time by sequentially illuminating or dulling the series of lights in proportion to the elapsed time, wherein a number of illuminated lights in the series of lights is indicative of an elapsed time within the challenge response interval;
determining, by the computing system, whether a user input action for satisfying the awareness challenge has been provided by the operator of the vehicle within the challenge response interval;
detecting, by the computing system, that the elapsed time has reached a safety time threshold, wherein the safety time threshold identifies an amount of elapsed time after expiration of the challenge response interval; and
in response to detecting the elapsed time corresponds to the safety time threshold, initiating, by the computing system, a safety stop routine.

2. The computer-implemented method of claim 1 , wherein one or more lights in the series of lights changes in a predefined order as the elapsed time approaches expiration of the challenge response interval.

3. The computer-implemented method of claim 1 , wherein the series of lights comprises multicolored lights.

4. The computer-implemented method of claim 1 , wherein the soft notification comprises an auditory stimulus that increases in one or more of volume or frequency in proportion to the elapsed time.

5. The computer-implemented method of claim 1 , wherein the changing visual stimulus is further indicative of expiration of the challenge response interval.

6. The computer-implemented method of claim 1 , comprising resetting, by the computing system, the awareness challenge in response to determining that the user input action for satisfying the awareness challenge has been provided by the operator of the vehicle within the challenge response interval.

7. The computer-implemented method of claim 1 , comprising communicating, by the computing system, a harsh notification indicative of the expiration of the challenge response interval in response to determining that the user input action for satisfying the awareness challenge has not been provided by the operator of the vehicle within the challenge response interval.

8. The computer-implemented method of claim 7 , wherein the soft notification comprises a visual stimulus and wherein the harsh notification comprises one or more of an auditory stimulus and a tactile stimulus.

9. The computer-implemented method of claim 1 , wherein the challenge response interval is randomized.

10. A computing system for generating a vehicle operator challenge, comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that store instructions for execution by the one or more processors to cause the computing system to perform operations, the operations comprising:
providing a prompt of an awareness challenge to an operator of a vehicle;
initiating a timer measuring an elapsed time from a start time of a challenge response interval;
communicating to the operator of the vehicle a soft notification indicative of the awareness challenge during the challenge response interval, wherein the soft notification includes a changing visual stimulus implemented by a series of lights that changes in proportion to the elapsed time by sequentially illuminating or dulling the series of lights in proportion to the elapsed time, wherein a number of illuminated lights in the series of lights is indicative of an elapsed time within the challenge response interval;
determining whether a user input action for satisfying the awareness challenge has been provided by the operator of the vehicle within the challenge response interval;
detecting that the elapsed time has reached a safety time threshold, wherein the safety time threshold identifies an amount of elapsed time after expiration of the challenge response interval; and
in response to detecting the elapsed time corresponds to the safety time threshold, initiating a safety stop routine.

11. The computing system of claim 10 , wherein one or more lights in the series of lights changes in a predefined order as the elapsed time approaches expiration of the challenge response interval.

12. The computing system of claim 10 , wherein the series of lights comprises multicolored lights.

13. The computing system of claim 10 , wherein the soft notification comprises an auditory stimulus that increases in one or more of volume or frequency in proportion to the elapsed time.

14. The computing system of claim 10 , the operations comprising communicating a harsh notification indicative of the expiration of the challenge response interval in response to determining that the user input action for satisfying the awareness challenge has not been provided by the operator of the vehicle within the challenge response interval.

15. The computing system of claim 14 , wherein the soft notification comprises a visual stimulus and wherein the harsh notification comprises one or more of an auditory stimulus and a tactile stimulus.

16. A vehicle, comprising:
one or more output devices;
one or more vehicle sensors;
one or more processors; and
one or more tangible, non-transitory, computer readable media that store instructions for execution by the one or more processors to cause the vehicle to perform operations, the operations comprising:
providing a prompt of an awareness challenge to an operator of the vehicle;
initiating a timer measuring an elapsed time from a start time of a challenge response interval;
communicating to the operator of the vehicle a soft notification indicative of the awareness challenge during the challenge response interval, wherein the soft notification includes a changing visual stimulus implemented by a series of lights that changes in proportion to the elapsed time by sequentially illuminating or dulling the series of lights in proportion to the elapsed time, wherein a number of illuminated lights in the series of lights is indicative of an elapsed time within the challenge response interval;
determining whether a user input action for satisfying the awareness challenge has been provided by the operator of the vehicle within the challenge response interval;
detecting that the elapsed time has reached a safety time threshold, wherein the safety time threshold identifies an amount of elapsed time after expiration of the challenge response interval; and
in response to detecting the elapsed time corresponds to the safety time threshold, initiating a safety stop routine.

17. The vehicle of claim 16 , wherein the series of lights comprises multicolored lights.

18. The vehicle of claim 16 , wherein the soft notification comprises an auditory stimulus that increases in one or more of volume or frequency in proportion to the elapsed time.

19. The vehicle of claim 16 , the operations comprising communicating a harsh notification indicative of expiration

Assignees

• Aurora Operations Inc

Inventors

• Donnelly Richard Brian
• Haban Philipp
• D'Eramo Christopher Matthew
• Staller Chad Jonathan
• Webb Clifford Shaun
• Dallesandro Douglas

Classifications

• B60K35/26

  using acoustic output · CPC title

• B60K35/10 Primary

  Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor · CPC title

• B60K35/22

  Display screens · CPC title

• B60K35/28

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title