Audio analytics and accessibility across applications and platforms

Patent metadata

Field	Value
Publication number	US-11579839-B1
Application number	US-202117462757-A
Country	US
Kind code	B1
Filing date	Aug 31, 2021
Priority date	Aug 31, 2021
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

Systems and methods for audio analytics and audio-based accessibility are provided. A user audio profile may be stored in memory for a user. The user audio profile may include a custom prioritization of one or more audio parameters associated with one or more audio modifications. Audio streams associated with a user device of the user may be monitored based on the user audio profile during a current session. The audio parameters may be detected as being present in the monitored audio streams, and the detected audio parameters may be prioritized based on the custom prioritization of the user audio profile. A sound property of at least one of the audio streams may be modified in real-time based on the prioritization of the detected audio parameters by applying the audio modifications of the user audio profile to the at least one audio stream before the at least one audio stream is provided to the user device.

First claim

What is claimed is:

1. A method for audio analytics and audio-based accessibility, the method comprising:
storing a user audio profile in memory for a user, wherein the user audio profile includes a custom prioritization of one or more audio parameters associated with one or more audio modifications;
monitoring one or more audio streams associated with a user device of the user during a current session, wherein monitoring the audio streams is based on the user audio profile;
monitoring data regarding sound in a real-world environment associated with the user as captured by a microphone associated with the user device;
detecting that one or more of the audio parameters are present in the monitored audio streams, wherein detecting the detected audio parameters is further based on the sound in the real-world environment;
prioritizing the detected audio parameters based on the custom prioritization of the user audio profile; and
modifying a sound property of at least one of the audio streams in real-time based on the prioritization of the detected audio parameters, wherein the sound property of the at least one audio stream includes at least one of volume, speed, and frequency response relative to one or more other ones of the audio streams, and wherein modifying the sound property includes applying the audio modifications of the user audio profile to the at least one audio stream before the at least one audio stream is provided to the user device.

2. The method of claim 1 , wherein monitoring the audio streams includes analyzing each of the audio streams to distinguish speech from other sounds, and wherein modifying the sound property includes boosting the sound property of the at least one audio stream that includes the speech relative to one or more other ones of the audio streams.

3. The method of claim 1 , wherein the detected audio parameters include speech by a plurality of different users, and wherein the custom prioritization includes a prioritization order of one or more user groups.

4. The method of claim 1 , wherein the sound in the real-world environment includes speech by the user, and wherein detecting that the audio parameters are present is based on identifying the speech by the user.

5. The method of claim 4 , wherein the user audio profile further includes data regarding a voice of the user, and further comprising providing the at least one modified audio stream to one or more other user devices associated with the current session.

6. The method of claim 1 , wherein the audio streams are associated with a plurality of different applications or platforms operating concurrently during the current session.

7. The method of claim 1 , wherein the audio modifications include at least one of adjusting one of the audio parameters, dampening, adjusting transparency, and noise cancellation.

8. A system for audio analytics and audio-based accessibility, the system comprising:
memory that stores a user audio profile in memory for a user, wherein the user audio profile includes a custom prioritization of one or more audio parameters associated with one or more audio modifications;
a communication interface that communicates over a communication network, wherein the communication interface receives one or more audio streams associated with a user device of the user during a current session; and
a processor that executes instructions stored in memory, wherein the processor executes the instructions to:
monitor the audio streams based on the user audio profile;
monitor data regarding sound in a real-world environment associated with the user as captured by a microphone associated with the user device;
detect that one or more of the audio parameters are present in the monitored audio streams, wherein detecting the detected audio parameters is further based on the sound in the real-world environment;
prioritize the detected audio parameters based on the custom prioritization of the user audio profile; and
modify a sound property of at least one of the audio streams in real-time based on the prioritization of the detected audio parameters, wherein the sound property of the at least one audio stream includes at least one of volume, speed, and frequency response relative to one or more other ones of the audio streams, and wherein modifying the sound property includes applying the audio modifications of the user audio profile to the at least one audio stream before the at least one audio stream is provided to the user device.

9. The system of claim 8 , wherein the processor monitors the audio streams by analyzing each of the audio streams to distinguish speech from other sounds, and wherein the processor modifies the sound property by boosting the sound property of the at least one audio stream that includes the speech relative to one or more other ones of the audio streams.

10. The system of claim 8 , wherein the detected audio parameters include speech by a plurality of different users, and wherein the custom prioritization includes a prioritization order of one or more user groups.

11. The system of claim 8 , wherein the sound in the real-world environment includes speech by the user, and wherein the processor detects that the audio parameters are present further based on identifying the speech by the user.

12. The system of claim 11 , wherein the user audio profile further includes data regarding a voice of the user, and wherein the communication interface further provides the at least one modified audio stream to one or more other user devices associated with the current session.

13. The system of claim 8 , wherein the audio streams are associated with a plurality of different applications or platforms operating concurrently during the current session.

14. The system of claim 8 , wherein the audio modifications include at least one of adjusting one of the audio parameters, dampening, adjusting transparency, and noise cancellation.

15. A non-transitory, computer-readable storage medium, having embodied thereon a program executable by a processor to perform a method for audio analytics and audio-based accessibility, the method comprising:
storing a user audio profile in memory for a user, wherein the user audio profile includes a custom prioritization of one or more audio parameters associated with one or more audio modifications;
monitoring one or more audio streams associated with a user device of the user during a current session, wherein monitoring the audio streams is based on the user audio profile;
monitoring data regarding sound in a real-world environment associated with the user as captured by a microphone associated with the user device;
detecting that one or more of the audio parameters are present in the monitored audio streams, wherein detecting the detected audio parameters is further based on the sound in the real-world environment;
prioritizing the detected audio parameters based on the custom prioritization of the user audio profile; and
modifying a sound property of at least one of the audio streams in real-time based on the prioritization of the detected audio parameters, wherein the sound property of the at least one audio stream includes at least one of volume, speed, and frequency response relative to one or more other ones of the audio streams, and wherein modifying the sound property includes applying the audio modifications of the user audio profile to the at least one audio stream before the at least one audio stream is provided to the user device.

Assignees

• Sony Interactive Entertainment LLC

Inventors

• Fortuna Ellana
• Karp Sarah
• Juenger Elizabeth

Classifications

• A63F13/79

  involving player-related data, e.g. identities, accounts, preferences or play histories · CPC title

• H04N21/4788

  communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} · CPC title

• A63F13/54

  involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall · CPC title

• H04L67/306

  User profiles · CPC title

• A63F13/424

  involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition · CPC title