Coordinating delivery of notifications to the driver of a vehicle to reduce distractions

Patent metadata

Field	Value
Publication number	US-10850746-B2
Application number	US-201816044437-A
Country	US
Kind code	B2
Filing date	Jul 24, 2018
Priority date	Jul 24, 2018
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

A notification control system (NCS) is configured to coordinate the delivery of notifications issued from a set of devices to the driver of a vehicle. The NCS registers all interactive devices residing in a vehicle and causes those devices to conform to specific directives generated by the NCS. The NCS analyzes and predicts the cognitive and emotional load on the driver and also estimates the cognitive and emotional impact on the driver potentially caused by any incoming notifications. The NCS then coordinates the delivery of notifications to the driver in a manner that avoids overwhelming the cognitive and/or emotional capacity of the driver. Accordingly, the NCS may avoid situations where device notifications cause the driver to unsafely divert attention away from driving.

First claim

What is claimed is:

1. A computer-implemented method for transmitting notifications to a driver of a vehicle, the method comprising:
determining a first mental load on the driver based on sensor data;
determining a second mental load on the driver, wherein the second mental load comprises a predicted mental load impact on the driver attributable to a first notification;
determining, based on the first mental load and the second mental load, that a mental load threshold will be exceeded; and
in response to determining that the mental load threshold will be exceeded:
changing a sensory modality of the first notification to generate a modified notification having a second predicted mental load impact on the driver lower than the predicted mental load impact on the driver attributable to the first notification; and
transmitting the modified notification to one or more output devices, wherein the one or more output devices output the modified notification within the vehicle.

2. The computer-implemented method of claim 1 , wherein the modified notification causes a third mental load on the driver that does not exceed the mental load threshold.

3. The computer-implemented method of claim 1 , further comprising:
determining a first cognitive load on the driver based on the sensor data;
determining a first emotional load on the driver based on the sensor data; and
determining the first mental load based on the first cognitive load and the first emotional load.

4. The computer-implemented method of claim 1 , wherein the first mental load comprises a cognitive load on the driver, and wherein the sensor data comprises at least one of pupillometry data, heart rate data, breathing rate data, or eye gaze data.

5. The computer-implemented method of claim 1 , wherein the first mental load comprises an emotional load on the driver, and wherein the sensor data comprises at least one of facial expression data, voice tone data, or galvanic skin response data.

6. The computer-implemented method of claim 1 , wherein the first mental load corresponds to a portion of a first time span, and the second mental load corresponds to another portion of the first time span and is associated with the first notification.

7. The computer-implemented method of claim 1 , further comprising delaying, in response to determining that the mental load threshold will be exceeded, a transmission time of the modified notification until after a first time span that corresponds to the first mental load.

8. The computer-implemented method of claim 1 , wherein the sensory modality of the first notification comprises a first encoding medium of first content included in the first notification, wherein changing the sensory modality comprises transcoding the first content from the first encoding medium to a second encoding medium, and wherein the modified notification is transmitted within a first time span that corresponds to the first mental load.

9. The computer-implemented method of claim 8 , wherein the first encoding medium comprises a visual medium and the second encoding medium comprises an auditory medium.

10. A non-transitory computer-readable medium storing program instructions that, when executed by a processor, cause the processor to transmit notifications to a driver of a vehicle by performing the steps of:
determining a first mental load on the driver based on sensor data;
determining a second mental load on the driver, wherein the second mental load comprises a predicted mental load impact on the driver attributable to a first notification;
determining, based on the first mental load and the second mental load, that a mental load threshold will be exceeded;
determining a priority level associated with the first notification; and
in response to determining that the mental load threshold will be exceeded:
changing a sensory modality of the first notification to generate a modified notification having a second predicted mental load impact on the driver lower than the predicted mental load impact on the driver attributable to the first notification; and
transmitting, based on the priority level, either the first notification or a modified version of the first notification to one or more output devices to be output within the vehicle, thereby causing a third mental load on the driver that does not exceed the mental load threshold.

11. The non-transitory computer-readable medium of claim 10 , wherein the step of determining the second mental load comprises:
identifying content included in the first notification; and
inputting the content to a machine learning model, wherein the machine learning model predicts physiologic responses of the driver to stimuli.

12. The non-transitory computer-readable medium of claim 11 , further comprising training the machine learning model based on a previous response of the driver to a previous notification and based on content included in the previous notification.

13. The non-transitory computer-readable medium of claim 11 , wherein the first notification is generated by a device integrated into the vehicle.

14. The non-transitory computer-readable medium of claim 11 , wherein the first notification is generated by a portable device or wearable device associated with the driver.

15. The non-transitory computer-readable medium of claim 11 , wherein the first notification is generated by a portable device associated with a passenger of the vehicle, and further comprising transmitting the first notification to the portable device associated with the passenger.

16. The non-transitory computer-readable medium of claim 11 , wherein the first notification is generated by a portable device associated with a passenger of the vehicle, and further comprising outputting the first notification within a sound field associated with the passenger, wherein the sound field excludes the driver.

17. The non-transitory computer-readable medium of claim 11 , further comprising registering a set of devices that generate notifications with a centralized controller that schedules the notifications to be output to the driver, wherein a first device included in the set of devices generates the first notification.

18. The non-transitory computer-readable medium of claim 11 , further comprising determining a transmission time associated with the first notification based on the priority level.

19. A system, comprising:
a sensor array that captures physiologic data associated with a driver of a vehicle;
a memory storing a software application; and
a processor that, when executing the software application, is configured to perform the steps of:
determining a first mental load on the driver based on sensor data,
determining a second mental load on the driver, wherein the second mental load comprises a predicted mental load impact on the driver attributable to a first notification,
determining, based on the first mental load and the second mental load, that a mental load threshold will be exceeded, and
in response to determining that the mental load threshold will be exceeded:
changing a sensory modality of the first notification to generate a modified notification having a second predicted mental load impact on the driver lower than the predicted mental load impact on the driver attributable to the first notification, and
transmitting the modified notification to one or more output devices, wherein the one or more output devices output the modified notification within the vehicle, and the modified notification causes a third mental load on the driver

Assignees

• Harman Int Ind

Inventors

• Marti Stefan
• Boulanger Adam
• Verbeke Joseph
• Di Censo Davide
• Spasojevic Mirjana
• Nahman Jaime Elliot

Classifications

• B60K35/28

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title

• B60K35/29

  Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions · CPC title

• B60K35/265

  Voice · CPC title

• B60K35/22 Primary

  Display screens · CPC title

• H04L51/226

  Delivery according to priorities · CPC title