Methods and vehicles for using determined mood of a human driver and moderating vehicle response

Patent metadata

Field	Value
Publication number	US-10535341-B2
Application number	US-201715470881-A
Country	US
Kind code	B2
Filing date	Mar 27, 2017
Priority date	Apr 22, 2011
Publication date	Jan 14, 2020
Grant date	Jan 14, 2020

Abstract

Methods and systems for determining a mood of a human driver of a vehicle and using the mood for generating a vehicle response, is provided. One example method includes capturing, by a camera of the vehicle, a face of the human driver. The capturing is configured to capture a plurality of images over a period of time, and the plurality of images are analyzed to identify a facial expression and changes in the facial expression of the human driver over the period of time. The method further includes capturing, by a microphone of the vehicle, voice input of the human driver. The voice input is captured over the period of time. The voice input is analyzed to identify a voice profile and changes in the voice profile of the human driver over the period of time. The method processes, by a processor of the vehicle, a combination of the facial expression and the voice profile captured during the period of time to predict the mood of the human driver. The method generates the vehicle response that is responsive to the mood of the human driver. The vehicle response is configured to make at least one adjustment to a setting of the vehicle. The adjustment is selected based on the mood of the human driver. The vehicle response can be used to make the driver more calm and/or assist in reducing distracted driving.

First claim

What is claimed is:

1. A method for determining a mood of a human driver of a vehicle and using the mood for generating a vehicle response, comprising,
capturing, by a camera of the vehicle, a face of the human driver, the capturing is configured to capture a plurality of images over a period of time, the plurality of images being analyzed to identify a facial expression and changes in the facial expression of the human driver over the period of time, the plurality of images further being analyzed for eye gaze patterns;
capturing, by a microphone of the vehicle, voice input of the human driver, the voice input is captured over the period of time, the voice input being analyzed to identify a voice profile and changes in the voice profile of the human driver over the period of time;
processing, by a processor of the vehicle, a combination of the facial expression, the eye gaze patterns and the voice profile captured during the period of time to determine the mood of the human driver; and
generating the vehicle response that is responsive to the voice input and the mood determined of the human driver, the vehicle response is configured to make at least one adjustment to a setting of the vehicle, the adjustment is selected based on the mood of the human driver.

2. The method of claim 1 , wherein said combination includes combining select features obtained from the facial expressions, the eye gaze patterns and the voice profile to build behavior models used for determining of the mood of the human driver, the determining includes a level of prediction of said mood.

3. The method of claim 1 , wherein facial characteristics of the human driver are related to a plurality of facial expressions that have been previously associated with specific moods, the facial characteristics include one or more of placement of the human driver's mouth, a way eyes are opened or closed by the human driver, cheekbone characteristics of the human driver, or biometric determining of features of the human driver, or a combination of two or more thereof.

4. The method of claim 1 , wherein processing the voice profile includes processing an audio sample of the voice input and identifying frequency and/or magnitude characteristics in the audio sample, the voice profile is associated with a tone of voice that is correlated to a possible mood among a plurality of moods.

5. The method of claim 1 , wherein facial characteristics of the human driver are related to a plurality of facial expressions that have been previously associated with specific moods, the facial characteristics include one or more of placement of the human driver's mouth, a way eyes are opened or closed by the human driver, cheekbone characteristics of the human driver, or biometric determining of features of the human driver, or a combination of two or more thereof and the processing the voice profile includes processing an audio sample of the voice input and identifying frequency and/or magnitude characteristics in the audio sample, the voice profile is associated with a tone of voice that is correlated to a possible mood among a plurality of moods.

6. The method of claim 5 , wherein the plurality of facial expressions are associated to a plurality of templates of facial characteristics, the plurality of templates being generated during a training or calibration session or generated over time based on learned facial characteristics, wherein learned facial characteristics are identified using one or more other detected physical or physiological conditions.

7. The method of claim 6 , wherein said physiological conditions include one or more of heart rate, or body heat, or blood pressure, or skin temperature, or skin galvanic resistance, or temperature of skin, or blood rush to a face of the human driver, or eye iris or retina changes, or odor/scent changes, or combinations of two or more thereof, and the physical conditions include one or more of a changes in the face of the human driver, or changes in a typing rhythm, or changes in gait, or changes in voice, or changes in voice tone, or voice inflections, or voice speed or slowness, or sharp voice harmonics, or voice identifiers or patterns, or elevated heat patterns, or palm veins, or eye iris or retina changes, or heat rate changes, or eye gaze motions, or combinations of two or more thereof.

8. The method of claim 1 , wherein the vehicle response to the mood includes one of waking up the human driver with a sound or air if the mood is a sleepy mood, or generating warnings and/or signals to alert the human driver, or generating recommendations to calm the human driver from an angry mood or agitated mood, or sending a notification to a third party indicating that the human driver is tired, or generating a notification to the human driver suggesting a temperature change, or generating an automatic temperature change based on the mood, or recommending the human driver stop driving when tired, or reducing recommendations when the determined mood is a rushed mood, or changing lighting of the vehicle automatically, or changing temperature of the vehicle automatically, or turn up or down a volume of music, or adjust a seat position, or a combination of two or more thereof, and
wherein at least one of said vehicle responses are predefined to reduce distracted driving or increase alertness of said human driver.

9. The method of claim 1 , wherein
the vehicle response is further based on current use of the vehicle, and use of the vehicle includes one or more of geolocation of the vehicle, speed of the vehicle, direction of the vehicle, a route of the vehicle, a time of day, occupants in the vehicle, historical use of the vehicle, online data of a user accessible to the vehicle or server, or combinations of two or more thereof, and
the vehicle response to the mood includes one of waking up the human driver with a sound or air if the mood is a sleepy mood, generating warnings and/or signals to alert the human driver, generating recommendations to calm the human driver from an angry mood or agitated mood, sending a notification to a third party indicating that the human driver is tired, generating a notification to the human driver suggesting a temperature change, or generating an automatic temperature change based on the mood, recommending the human driver stop driving when tired, reducing recommendations when the human driver when the determined mood is a rushed mood, or a combination of two or more thereof.

10. The method of claim 1 , wherein a user account with a profile for the human driver of the vehicle is provided, wherein the user account of the human driver is identified based on processing of credential data obtained from the human driver, the credential data used by electronics of the vehicle to associated the human driver to the user account and to the profile, the electronics of the vehicle configured to interface over a network with a server for use of the profile and associated updates to the profile as made via use of the vehicle or updates associated with learned behavior.

11. The method of claim 1 , wherein the mood includes one or more of a normal mood, or a frustrated mood, or an agitated mood, or an upset mood, or a hurried mood, or an urgency mood, or a rushed mood, or a stressed mood, or a calm mood, or a passive mood, or a sleepy mood, or a happy mood, or an excited mood, or combinations of two or more thereof, and
said mood is configured to change over time based on said captured face and voice input of the human driver.

12. The method of claim 1 , wherein the processing of the voice input includes identifying a dialect of the human driver, the dialect includes one or more of a language dial

Assignees

• Penilla Angel A
• Penilla Albert S
• Emerging Automotive Llc

Inventors

• Penilla Angel A
• Penilla Albert S

Classifications

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G10L25/57

  for processing of video signals · CPC title

• B60R16/0373

  Voice control (in general G10L) · CPC title

• G10L15/063

  Training · CPC title

• G10L15/25

  using position of the lips, movement of the lips or face analysis · CPC title