In-vehicle acoustic monitoring system for driver and passenger

Patent metadata

Field	Value
Publication number	US-11897331-B2
Application number	US-202117149585-A
Country	US
Kind code	B2
Filing date	Jan 14, 2021
Priority date	Jan 14, 2021
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

Sound signals that are not audible are generated by one or more speakers disposed in the vehicle. Reflected sound signals from a driver or a passenger of the vehicle that are not audible are received by one or more microphones disposed in the vehicle. A behavior-induced acoustic pattern is detected based on the reflected ultrasound signals. The behavior-induced acoustic pattern is analyzed to recognize a behavior of the driver or the passenger of the vehicle. A response or an alert is generated according to the recognized behavior of the driver or the passenger in the vehicle.

First claim

What is claimed is:

1. A computer-implemented method for operating a vehicle, the method comprising:
generating sound signals using one or more speakers disposed within the vehicle, wherein the sound signals are not audible;
receiving reflected sound signals from a driver or a passenger of the vehicle by one or more microphones disposed within the vehicle;
detecting a behavior-induced acoustic pattern based on the reflected sound signals received by the one or more microphones;
analyzing the behavior-induced acoustic pattern to determine a behavior of the driver or the passenger of the vehicle; and
generating an alert according to the behavior of the driver or the passenger of the vehicle in view of a set of one or more rules.

2. The method of claim 1 , wherein the sound signals comprise ultrasound signals.

3. The method of claim 1 , wherein the one or more microphones are disposed at an upper front left corner, an upper front right corner, between a driver seat and a front passenger seat, a back of the driver seat or the front passenger seat, or between two back passenger seats in the vehicle.

4. The method of claim 1 , wherein analyzing the behavior-induced acoustic pattern to determine a behavior of the driver or the passenger of the vehicle comprises matching the behavior-induced acoustic pattern with a list of predetermined acoustic patterns representing a list of predetermined behaviors.

5. The method of claim 1 , wherein detecting a behavior-induced acoustic pattern based on the reflected sound signals includes extracting a power spectrogram based on the reflected sound signals.

6. The method of claim 5 , wherein analyzing the behavior-induced acoustic pattern to determine a behavior of the driver or the passenger of the vehicle comprises comparing the power spectrogram with a predetermined power spectrogram associated with a particular behavior.

7. The method of claim 1 , further comprising determining a risk level of the behavior of the driver or the passenger, and wherein the alert is further generated based on the risk level of the behavior of the driver or the passenger.

8. The method of claim 7 , wherein generating the alert according to the risk level of the driver or the passenger includes, when the behavior is from the driver,
generating an alarm and activating an automatic driving mode to slow down to stop at a road side in response to the determined risk level being higher than a predetermined risk threshold.

9. The method of claim 7 , wherein generating the alert according to the risk level of the driver or the passenger includes, when the behavior is from the passenger and the vehicle is in an autonomous driving mode, reducing a speed of the vehicle or switching to a comfort driving mode of the autonomous driving mode.

10. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations of operating a vehicle, the operations comprising:
generating sound signals using one or more speakers disposed within the vehicle, wherein the sound signals are not audible;
receiving reflected sound signals from a driver or a passenger of the vehicle by one or more microphones disposed within the vehicle;
detecting a behavior-induced acoustic pattern based on the reflected sound signals received by the microphones;
analyzing the behavior-induced acoustic pattern to determine a behavior of the driver or the passenger of the vehicle; and
generating an alert according to the behavior of the driver or the passenger of the vehicle in view of a set of one or more rules.

11. The machine-readable medium of claim 10 , wherein the sound signals comprise ultrasound signals.

12. The machine-readable medium of claim 10 , wherein the one or more microphones are disposed at an upper front left corner, an upper front right corner, between a driver seat and a front passenger seat, a back of the driver seat or the front passenger seat, or between two back passenger seats in the vehicle.

13. The machine-readable medium of claim 10 , wherein analyzing the behavior-induced acoustic pattern to determine a behavior of the driver or the passenger of the vehicle comprises matching the behavior-induced acoustic pattern with a list of predetermined acoustic patterns representing a list of predetermined behaviors.

14. The machine-readable medium of claim 10 , wherein detecting a behavior-induced acoustic pattern based on the reflected sound signals includes extracting a power spectrogram based on the sound signals.

15. The machine-readable medium of claim 14 , wherein analyzing the behavior-induced acoustic pattern to determine a behavior of the driver or the passenger of the vehicle comprises comparing the power spectrogram with a predetermined power spectrogram associated with a particular behavior.

16. The machine-readable medium of claim 10 , wherein the operations further comprise determining a risk level of the behavior of the driver or the passenger, and wherein the alert is further generated based on the risk level of the behavior of the driver or the passenger.

17. The machine-readable medium of claim 16 , wherein generating the alert according to the risk level of the driver or the passenger includes, when the behavior is from the driver,
generating an alarm and activating an automatic driving mode to slow down to stop at a road side in response to the risk level being higher than a predetermined risk threshold.

18. The machine-readable medium of claim 16 , wherein generating the alert according to the risk level of the driver or the passenger includes, when the behavior is from the passenger and the vehicle is in an autonomous driving mode, reducing a speed of the vehicle or switching to a comfort driving mode of the autonomous driving mode.

19. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations of operating a vehicle, the operations including
generating sound signals using one or more speakers disposed within the vehicle, wherein the sound signals are not audible,
receiving reflected sound signals from a driver or a passenger of the vehicle by one or more microphones disposed within the vehicle,
detecting a behavior-induced acoustic pattern based on the reflected sound signals received by the microphones,
analyzing the behavior-induced acoustic pattern to determine a behavior of the driver or the passenger of the vehicle, and
generating an alert according to the behavior of the driver or the passenger of the vehicle in view of a set of one or more rules.

20. The system of claim 19 , wherein the sound signals comprise ultrasound signals.

Assignees

• Baidu Usa Llc

Inventors

• Wang Wei
• Luo Qi
• Xu Kecheng
• Sun Hongyi
• Reynolds Wesley
• Lin Zejun

Classifications

• B60K28/02 Primary

  responsive to conditions relating to the driver · CPC title

• B60W30/182

  Selecting between different operative modes, e.g. comfort and performance modes · CPC title

• B60W50/14 Primary

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60W60/0013

  specially adapted for occupant comfort · CPC title

• B60W60/0015

  specially adapted for safety · CPC title