Audio based system and method for in-vehicle context classification

Patent metadata

Field	Value
Publication number	US-9311930-B2
Application number	US-201414165902-A
Country	US
Kind code	B2
Filing date	Jan 28, 2014
Priority date	Jan 28, 2014
Publication date	Apr 12, 2016
Grant date	Apr 12, 2016

Abstract

A method of determining contexts for a vehicle, each context corresponding to one or more events associated with the vehicle, for example that the radio is on and a window is open. The method comprises detecting sound activities in an audio signal captured in the vehicle, and assigning context to the vehicle based on the detected sound activities. Non-audio data such as the operational status of a vehicle system or device is used to help assign contexts.

First claim

The invention claimed is:

1. A method of determining contexts and associated sound activities for a vehicle, the method including:
associating a plurality of vehicle contexts with a respective one or more of a plurality of sound activities;
detecting an audio signal in the vehicle;
detecting, based on said detected audio signal, at least one of said sound activities in said audio signal;
subsequent to detecting at least one of said sound activities, identifying a vehicle context associated with said detected at least one of said sound activities;
providing a respective n-gram model for each of said vehicle contexts; and
assigning to said vehicle said at least one identified vehicle context and said detected at least one of said sound activities with which said identified vehicle context is associated, wherein said assigning involves using a history of at least one previously assigned vehicle context together with said n-gram models in identifying said at least one of said vehicle contexts.

2. The method of claim 1 , wherein said assigning involves using non-audio vehicle data in determining said at least one of said vehicle contexts.

3. The method of claim 2 , wherein said non-audio vehicle data comprises data indicating the operational status of one or more of the vehicle's systems or vehicle's devices.

4. The method of claim 2 , including obtaining said non-audio data from at least one vehicle sensor.

5. The method of claim 4 , wherein said at least one sensor is configured to detect the status of any one or more aspects of the vehicle, including a windshield wiper, direction indicator, media player, navigation system, window, sun roof, rain sensor, fan, air conditioning system or telephone system.

6. The method of claim 2 , including obtaining said non-audio data from a vehicle control system.

7. The method of claim 6 , including obtaining said non-audio data from a control unit of the vehicle.

8. The method of claim 2 , wherein said using non-audio vehicle data in determining said at least one of said vehicle contexts involves using said non-audio vehicle data to determine compatibility of at least some of said vehicle contexts with said detected audio signal.

9. The method of claim 1 , including detecting said audio signal using at least one microphone.

10. The method of claim 9 , wherein said microphone is incorporated into said vehicle such that said audio signal corresponds to sounds in a cabin of the vehicle detected by said at least one microphone.

11. The method of claim 1 , including segmenting said audio signal into audio segments, wherein said detecting said at least one of said sound activities involves detecting a respective at least one of said sound activities in each audio segment; and said assigning involves assigning said a respective at least one of said identified vehicle contexts and said at least one of said associated sound activities in respect of each audio segment.

12. The method of claim 11 , including using non-audio vehicle data in determining boundaries for the audio segments during the segmentation process.

13. The method of claim 11 , including performing feature extraction on said audio segments to provide a respective frequency-based definition of each audio segment.

14. The method of claim 13 , including assuming that temporal sparsity exists in said respective frequency-based definition.

15. The method of claim 11 , including organizing said audio segments into respective frames, wherein each frame corresponds to a single sound activity or sound activity model.

16. The method of claim 11 , wherein said assigning involves using non-audio vehicle data to determine compatibility of at least some of said vehicle contexts with each audio segment, and wherein said detecting a respective at least one of said sound activities in each audio segment involves detecting sound activities that are associated with said vehicle contexts that are determined to be compatible with said detected audio segment.

17. The method of claim 11 , wherein said assigning involves using non-audio vehicle data to determine compatibility of at least some of said vehicle contexts with each audio segment, and wherein said assigning said a respective at least one of said vehicle contexts in respect of each audio segment involves assigning sound activities that are associated with said vehicle contexts that are determined to be compatible with said detected audio segment.

18. The method of claim 11 , including providing a plurality of sound activity models, wherein each of said plurality of sound activity models comprises a mathematical representation of a respective one or more of said sound activities, and wherein said detecting at least one of said sound activities in said audio signal involves comparing said audio segments against at least some of said sound activity models.

19. The method of claim 1 , including providing a plurality of sound activity models, wherein each of said plurality of sound activity models comprises a mathematical representation of a respective one or more of said sound activities, and wherein said detecting at least one of said sound activities in said detected audio signal involves comparing said audio signal against at least some of said plurality of sound activity models.

20. The method of claim 19 , wherein said associating a plurality of vehicle contexts with a respective one or more of said sound activities involves associating each said plurality of vehicle contexts with a respective one or more of said sound activity models corresponding to said respective one or more of said sound activities.

21. The method of claim 19 , wherein said assigning involves using non-audio vehicle data to determine compatibility of at least some of said vehicle contexts with said detected audio signal, and wherein said comparing said detected audio signal against at least some of said plurality of sound activity models involves comparing sound activity models that are associated with said vehicle contexts that are determined to be compatible with said detected audio signal.

22. The method of claim 19 , wherein said comparing said detected audio signal against at least some of said plurality of sound activity models involves computing a respective matching score for at least some of said sound activity models, comparing said matching scores, and wherein said detecting at least one of said sound activities in said audio signal involves determining which of said sound activities is detected based on said comparison of said matching scores.

23. The method of claim 22 , including segmenting said detected audio signal into audio segments, wherein said detecting said at least one of said sound activities involves detecting a respective at least one of said sound activities in each audio segment; and said assigning involves assigning a respective at least one of said vehicle contexts in respect of each audio segment, and wherein said comparing said audio signal against at least some of said sound activity, said computing a respective matching score for at least some of said plurality of sound activity models, and said determining which of said sound activities is detected are performed in respect of each audio segment.

24. The method of claim 22 , wherein said comparing said matching scores involves weighting said matching scores using a respective n-gram model of the respective vehicle context associated with at a respective one of the

Assignees

• Qualcomm Technologies Int Ltd

Inventors

• Srinivasan Ramji
• Rea Derrick
• Trainor David

Classifications

• G10L25/72

  for transmitting results of analysis · CPC title

• G10L25/51

  for comparison or discrimination · CPC title

• G10L25/27 Primary

  characterised by the analysis technique · CPC title

• H04R2499/13

  Acoustic transducers and sound field adaptation in vehicles · CPC title