In-cabin acoustic-based passenger occupancy and situation state assessment

Patent metadata

Field	Value
Publication number	US-11501584-B2
Application number	US-202016830769-A
Country	US
Kind code	B2
Filing date	Mar 26, 2020
Priority date	Mar 26, 2020
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

Techniques are disclosed to use existing vehicle speakers alone or in conjunction with other sensors (e.g. SRS sensors and/or microphones) that may already be implemented as part of the vehicle to identify acoustic signatures. Suitable low-cost and widely available hardware components (e.g., relays) may be used to modify the vehicle's existing speakers for a bi-directional mode of operation. Moreover, the vehicle's existing of audio amplifiers may be used to amplify signals collected by the speakers when operating in “reverse,” and process these collected signals to determine vehicle state information.

First claim

What is claimed is:

1. A device for analyzing acoustic data in a vehicle, comprising:
a processor configured to receive, from at least one speaker in the vehicle, acoustic data signals generated from vibration of the at least one speaker configured as a microphone; and
a memory configured to store instructions that, when executed by the processor, cause the processor to calculate an acoustic signature associated with the at least one speaker based on the received acoustic data signals, and to determine a state of the vehicle based on the acoustic signature,
wherein the processor is further configured to provide the state of the vehicle to an interface for communication with a control system of the vehicle to execute an action based upon the state of the vehicle,
wherein the at least one speaker is (i) part of an audio sound system in the vehicle, and (ii) selectively configurable as a microphone to generate acoustic data signals and as a speaker to emit sound in accordance with the audio sound system, and
wherein the acoustic signature represents a frequency response of the at least one speaker when configured as a microphone that indicates changes in acoustic amplitude measured by the at least one speaker over an acoustic frequency spectrum.

2. The device of claim 1 , wherein the acoustic data signals are generated by the at least one speaker configured as a microphone from at least one of (i) vibrations caused by driving the vehicle, and (ii) vibrations resulting from an engine of the vehicle, and
wherein the state of the vehicle includes at least one of (i) a road condition, and (ii) vehicle diagnostics.

3. The device of claim 1 , wherein the state of the vehicle includes one or more of (i) an occupancy state that is indicative of which vehicle seats are occupied by a person or object, (ii) an open or closed state of one or more vehicle doors, and (iii) an open or closed state of one or more vehicle windows or sunroofs.

4. The device of claim 1 , wherein:
the at least one speaker comprises a first speaker and a second speaker,
the processor is configured to drive the second speaker with a predetermined sound, and
the acoustic data signals are generated from the first speaker in response to the predetermined sound generated from the second speaker.

5. The device of claim 4 , wherein the processor is configured to determine that an object is disposed in the vehicle between the first speaker and the second speaker when the acoustic signature associated with the first speaker deviates from a predetermined frequency response at one or more spectral locations in excess of a threshold value.

6. The device of claim 1 , wherein the at least one speaker comprises a set of speakers in the vehicle, and
wherein the instructions stored in the memory, when executed by the processor, cause the processor to receive acoustic data from each one of the set of speakers, to calculate a respective acoustic signature for each speaker from among the set of speakers, and to classify the acoustic signature received from each speaker from among the set of speakers in accordance with a machine learning algorithm to determine the state of the vehicle.

7. The device of claim 6 , wherein the machine learning algorithm is trained in accordance with training data that includes a frequency response for each speaker from among the set of speakers for a plurality of different vehicle states, the plurality of different vehicle states including one or more combinations of (i) different vehicle seats being occupied by a person or object, (ii) different vehicle doors being opened or closed, and (iii) different vehicle windows or sunroofs being opened or closed.

8. A vehicle, comprising:
a first speaker in the vehicle configured to generate acoustic data signals from a vibration of the first speaker configured as a microphone; and
local processing circuitry configured to calculate an acoustic signature associated with the first speaker based on the received acoustic data signals, and to determine a state of the vehicle based on the acoustic signature,
wherein the local processing circuitry is further configured to provide the state of the vehicle to an interface for communication with a control system of the vehicle to execute an action based upon the state of the vehicle,
wherein the first speaker is (i) part of an audio sound system in the vehicle, and (ii) selectively configurable as a microphone to generate acoustic data signals and as a speaker to emit sound in accordance with the audio sound system, and
wherein the acoustic signature represents a frequency response of the first speaker when configured as a microphone that indicates changes in acoustic amplitude measured by the first speaker over an acoustic frequency spectrum.

9. The vehicle of claim 8 , wherein the acoustic data signals are generated by the first speaker configured as a microphone from at least one of (i) vibrations caused by driving the vehicle, and (ii) vibrations resulting from an engine of the vehicle, and
wherein the state of the vehicle includes at least one of (i) a road condition, and (ii) vehicle diagnostics.

10. The vehicle of claim 8 , wherein the state of the vehicle includes one or more of (i) an occupancy state that is indicative of which vehicle seats are occupied by a person or object, (ii) an open or closed state of one or more vehicle doors, and (iii) an open or closed state of one or more vehicle windows or sunroofs.

11. The vehicle of claim 8 , wherein the local processing circuitry is configured to drive a second speaker with a predetermined sound, and
wherein the acoustic data signals are generated by the first speaker in response to the predetermined sound generated by the second speaker.

12. The vehicle of claim 11 , wherein the local processing circuitry is configured to determine that an object is disposed in the vehicle between the first speaker and the second speaker when the acoustic signature associated with the first speaker deviates from a predetermined frequency response at one or more spectral locations in excess of a threshold value.

13. The vehicle of claim 8 , wherein the first speaker is from among a set of speakers in the vehicle, and
wherein the local processing circuitry is configured to receive acoustic data from each speaker from among the set of speakers, to calculate a respective acoustic signature for each speaker from among the set of speakers, and to classify the acoustic signature received from each speaker from among the set of speakers in accordance with a machine learning algorithm to determine the state of the vehicle.

14. The vehicle of claim 13 , wherein the machine learning algorithm is trained in accordance with training data that includes a frequency response for each speaker from among the set of speakers for a plurality of different vehicle states, the plurality of different vehicle states including one or more combinations of (i) different vehicle seats being occupied by a person or object, (ii) different vehicle doors being opened or closed, and (iii) different vehicle windows or sunroofs being opened or closed.

15. A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors of a control system associated with a vehicle, cause the control system to:
receive, from a first speaker in the vehicle, acoustic data signals generated from a vibration of the first speaker configured as a microphone;
calculate an acoustic signature associated with the first speaker based on the received acoustic data signals;
determine a state of the vehicle based on t

Assignees

• Intel Corp

Inventors

• Zamora Esquivel Julio
• Alvarez Ignacio J
• Cordourier Maruri Hector
• Cruz Vargas Jesus
• Gonzalez Aguirre David I
• Ibarra Von Borstel Alejandro
• Lopez Meyer Paulo
• Camacho Perez Jose Rodrigo

Classifications

• G06N3/045

  Combinations of networks · CPC title

• B60R11/0217

  for loud-speakers · CPC title

• G10L25/51

  for comparison or discrimination · CPC title

• G06N3/08

  Learning methods · CPC title

• B60R11/0247

  for microphones or earphones · CPC title