Context aware hearing optimization engine

Patent metadata

Field	Value
Publication number	US-11501772-B2
Application number	US-202016780825-A
Country	US
Kind code	B2
Filing date	Feb 3, 2020
Priority date	Sep 30, 2016
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

One or more context aware processing parameters and an ambient audio stream are received. One or more sound characteristics associated with the ambient audio stream are identified using a machine learning model. One or more actions to perform are determined using the machine learning model and based on the one or more context aware processing parameters and the identified one or more sound characteristics. The one or more actions are performed.

First claim

What is claimed is:

1. A method comprising,
receiving one or more context aware processing parameters, an ambient audio stream and a secondary audio stream, the one or more context aware processing parameters comprising one or more parameters indicating a user's current contextual state, the one or more parameters including at least one parameter corresponding to a location or a situation of the user;
identifying, using one or more machine learning models, one or more sound characteristics associated with the ambient audio stream;
determining, using the one or more machine learning models, one or more actions to perform based, at least in part, on the one or more context aware processing parameters and one or more identified sound characteristics, the one or more actions comprising applying one or more audio processing parameters to the ambient audio stream, the secondary audio stream, or both the ambient audio stream and the secondary audio stream, the one or more actions corresponding to personalized action sets for the user, each of the personalized action sets being associated with at least one indicator indicating conditions or circumstances in which an associated personalized action set is appropriate for selection, the at least one indicator including global contextual state information corresponding with a state of at least one other user's personal audio system;
wherein the global contextual state information comprises an amalgamation of the contextual states of a plurality of other users; and
performing the one or more actions.

2. The method of claim 1 , wherein the at least one parameter corresponds to the location of the user and indicates a room in which the user is currently located.

3. The method of claim 1 , wherein identifying the one or more sound characteristics associated with the ambient audio stream involves detecting a baby's cry.

4. The method of claim 3 , wherein the one or more actions involves reducing an amount of baby cries heard by the user.

5. The method of claim 3 , wherein the one or more actions involves adjusting a setting of at least one microphone.

6. The method of claim 5 , wherein adjusting the setting of the at least one microphone involves initiating or modifying one or more beam forming parameters.

7. The method of claim 1 , wherein the one or more audio processing parameters correspond to one or more of filtering, equalization, compression, limiting, echo cancellation or noise reduction.

8. The method of claim 1 , wherein the secondary audio stream includes audio from an application being executed by a local device.

9. The method of claim 8 , wherein the audio from the application corresponds to one or more of directions from a navigation app, music from a streaming music app, information from a self-guided audio tour app or a sportscaster describing a sporting event.

10. The method of claim 1 , wherein the at least one indicator includes one or more of an ambient sound profile or context information.

11. The method of claim 1 , further comprising retrieving the personalized action sets and the at least one indicator from a contextual state memory.

12. The method of claim 1 , further comprising receiving sensor data from at least one sensor, wherein determining the one or more actions to perform is based, at least in part, on the sensor data.

13. The method of claim 1 , further comprising extracting conversation data from an ambient audio stream stored in an audio snippet memory, wherein determining the one or more actions to perform is based, at least in part, on the conversation data.

14. The method of claim 1 , wherein the at least one parameter corresponds to the situation of the user, further comprising determining, using the one or more machine learning models, that the one or more sound characteristics associated with the ambient audio stream correspond with the situation of the user.

15. The method of claim 14 , wherein the one or more machine learning models determine that the one or more sound characteristics associated with the ambient audio stream match an ambient sound profile corresponding to the situation of the user.

16. The method of claim 15 , wherein the situation of the user includes one or more of an airplane cabin situation, a dinner situation, a concert situation, a subway situation, an urban street situation, a siren situation or a crying baby situation.

17. The method of claim 1 , wherein at least one of the one or more parameters corresponds to the location of the user and indicates a limited geographic area that is defined by one or more of a center point and a radius, a pair of coordinates identifying diagonal corners of a rectangular area or a series of coordinates identifying vertices of a polygon.

18. The method of claim 1 , wherein the global contextual state information is downloaded dynamically based upon one or more factors.

19. The method of claim 18 , wherein the one or more factors include one or more of an audio characteristic of ambient audio, a location of a personal audio system, an action set of one or more other users corresponding to similar contextual information, or time-based data.

20. The method of claim 1 , further comprising downloading a global contextual state that matches the ambient sound profile and associated context information responsive to determining that the ambient sound profile and associated context information is not associated with an action set.

21. The method of claim 1 , wherein the determining involves determining whether a strength of recommendation corresponding to a personalized action set is equal to or above a recommendation threshold.

22. The method of claim 21 , wherein the determining involves selecting a personalized action set from a data structure that includes a plurality of personalized action sets, each personalized action set of the plurality of personalized action sets having corresponding ambient sound profile information and corresponding strength of recommendation information.

23. The method of claim 21 , wherein a personalized action set is selected if the corresponding strength of recommendation is equal to or above the recommendation threshold.

24. The method of claim 21 , wherein it is determined that a plurality of personalized action sets have a corresponding strength of recommendation that is equal to or above the recommendation threshold, further comprising selecting a personalized action set having a corresponding higher recommendation threshold or a corresponding highest recommendation threshold.

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Meacham Jacob
• Sills Matthew
• Lanman Iii Richard Fritz
• Baker Jeffrey

Classifications

• G10L25/30

  using neural networks · CPC title

• H04R2410/05

  Noise reduction with a separate noise microphone · CPC title

• G10L25/84

  for discriminating voice from noise · CPC title

• H04R2225/41

  Detection or adaptation of hearing aid parameters or programs to listening situation, e.g. pub, forest · CPC title

• G10L2021/02166

  Microphone arrays; Beamforming · CPC title