Adaptive infotainment system based on vehicle surrounding and driver mood and/or behavior

Patent metadata

Field	Value
Publication number	US-10170111-B2
Application number	US-201715422024-A
Country	US
Kind code	B2
Filing date	Feb 1, 2017
Priority date	Jan 19, 2017
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

A vehicular infotainment system, a vehicle and a method of controlling interaction between a vehicle driver and an infotainment system. A multimedia device, a human-machine interface and sensors are used to collect in-vehicle driver characteristic data and extra-vehicle driving conditions. The system additionally includes—or is otherwise coupled to—a computer to convert one or both of traffic pattern data and vehicular positional data into a driver elevated cognitive load profile. In addition, the computer converts the driver characteristic data into a driver mood profile. The system can process these profiles to selectively adjust one or both of the amount of time needed to accept audio commands from a driver and the amount of time needed to provide an audio response to the driver in situations where the system determines the presence of at least one of the elevated cognitive load and a driver mood.

First claim

What is claimed is:

1. A vehicular infotainment system comprising:
a multimedia device comprising a human-machine interface and at least one of an audio player and a video player;
a plurality of sensors for acquiring driver characteristic data and at least one of traffic pattern data and vehicular positional data; and
a computer signally cooperative with the sensors and multimedia device and comprising a processor and a memory signally coupled to one another such that program instructions stored in the memory and executable by the processor:
cause the computer to convert the acquired at least one of traffic pattern data and vehicular positional data into a driver elevated cognitive load profile;
cause the computer to convert a first portion of the driver characteristic data that is input into the multimedia device from the group consisting of voice speed, voice volume, voice frequency and facial expression into a baseline driver mood profile;
cause the computer to convert a second portion of the driver characteristic data that is input into the multimedia device from the group consisting of voice speed, voice volume, voice frequency and facial expression into an instant driver mood profile wherein at least one of the second portion of the driver characteristic data is quantitatively different from a respective one of the first portion of the driver characteristic data;
cause the computer to selectively adjust the amount of time that the human-machine interface accepts the second portion of the driver characteristic data from a driver in situations where a determination is made by the computer of at least one of the elevated cognitive load profile and a difference between the instant driver mood profile and the baseline driver mood profile; and
cause the computer to selectively adjust at least one of an audio response and a video response from the multimedia device to a driver in situations where a determination is made by the computer of at least one of the elevated cognitive load profile and a difference between the instant driver mood profile and the baseline driver mood profile, wherein when the computer selectively adjusts at least one of the audio response and the video response from the multimedia device to a driver, it refrains from providing the human-machine interface response at least until such time as the determination by the computer indicates that the respective at least one of the elevated cognitive load profile and the difference between the instant driver mood profile and the baseline driver mood profile is no longer present.

2. The system of claim 1 , wherein the human machine interface comprises at least one of a microphone and a camera with which to acquire the driver characteristic data.

3. The system of claim 1 , wherein the quantitatively different driver mood profile corresponds to a mood indicative of driver impatience.

4. The system of claim 1 , wherein the plurality of sensors for acquiring at least one of traffic pattern data and vehicular positional data further comprises at least one sensor for acquiring at least one of a braking pattern and accelerating pattern.

5. The system of claim 1 , wherein the plurality of sensors for acquiring at least one of traffic pattern data and vehicular positional data is selected from the group consisting of a GPS sensor, a visual sensor, a radar sensor, a lidar sensor and an acoustic sensor.

6. The system of claim 1 , wherein the computer is further signally cooperative with a driver calendar such that driver meeting schedule data contained therein is conveyed to the computer to convert such schedule data into an additional parameter for the driver elevated cognitive load profile.

7. The system of claim 1 , wherein when the computer selectively adjusts at least one of the audio response and the video response from the multimedia device to a driver, it slows down the human-machine interface response.

8. The system of claim 1 , wherein when the computer selectively adjusts at least one of the audio response and the video response from the multimedia device to a driver, it speeds up the human-machine interface response.

9. The system of claim 1 , wherein when the computer selectively adjusts at least one of the audio response and the video response from the multimedia device to a driver, it provides a different audio feedback to a driver command than that given in response to a driver command where neither the instant driver mood profile or the elevated cognitive load profile are present.

10. The system of claim 1 , wherein the at least one of the audio response and the video response from the multimedia device to a driver is aborted once both the elevated cognitive load profile and the difference between the instant driver mood profile and the baseline driver mood profile is present.

11. A vehicle comprising:
a platform comprising a wheeled chassis defining passenger compartment and an engine compartment comprising an engine disposed therein,
a guidance apparatus cooperative with the wheeled chassis and the engine in order to provide motive control; and
an infotainment system for processing voice input, the infotainment system comprising:
a multimedia device comprising a human-machine interface and at least one of an audio player and a video player;
a plurality of sensors for acquiring driver characteristic data and at least one of traffic pattern data and vehicular positional data; and
a computer signally cooperative with the sensors and multimedia device and comprising a processor and a memory signally coupled to one another such that program instructions stored in the memory and executable by the processor:
cause the computer to convert the acquired at least one of traffic pattern data and vehicular positional data into a driver elevated cognitive load profile;
cause the computer to convert a first portion of the driver characteristic data that is input into the multimedia device from the group consisting of voice speed, voice volume, voice frequency and facial expression into a baseline driver mood profile;
cause the computer to convert a second portion of the driver characteristic data that is input into the multimedia device from the group consisting of voice speed, voice volume, voice frequency and facial expression into an instant driver mood profile wherein at least one of the second portion of the driver characteristic data is quantitatively different from a respective one of the first portion of the driver characteristic data;
cause the computer to selectively adjusts the amount of time that the human-machine interface accepts the second portion of the driver characteristic data from a driver in situations where a determination is made by the computer of at least one of the elevated cognitive load profile and a difference between the instant driver mood profile and the baseline driver mood profile; and
cause the computer to selectively adjust at least one of an audio response and a video response from the multimedia device to a driver in situations where a determination is made by the computer of at least one of the elevated cognitive load profile and a difference between the instant driver mood profile and the baseline driver mood profile, wherein when the computer selectively adjusts at least one of the audio response and the video response from the multimedia device to a driver, it refrains from providing the human-machine interface response at least until such time as the determination by the computer indicates that the respective at least one of the elevated cognitive load profile and the difference between the instant driver mood profile and the baseline driver mood profile is no longer present.

Assignees

• Toyota Eng & Mfg North America

Inventors

• Puranik Nishikant Narayan

Classifications

• H04N21/44218

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/24

  Speech recognition using non-acoustical features · CPC title

• B60K35/00

  Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles · CPC title

• G06F3/048

  Interaction techniques based on graphical user interfaces [GUI] · CPC title