Systems and methods for automated customized voice filtering

Patent metadata

Field	Value
Publication number	US-12230288-B2
Application number	US-202217828116-A
Country	US
Kind code	B2
Filing date	May 31, 2022
Priority date	May 31, 2022
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Systems and methods for audio processing are described. An audio processing system receives audio content that includes a voice sample. The audio processing system analyzes the voice sample to identify a sound type in the voice sample. The sound type corresponds to pronunciation of at least one specified character in the voice sample. The audio processing system generates a filtered voice sample at least in part by filtering the voice sample to modify the sound type. The audio processing system outputs the filtered voice sample.

First claim

What is claimed is:

1. An apparatus for audio processing, the apparatus comprising:
at least one memory storing instructions; and
at least one processor that executes the instructions, wherein execution of the instructions by the at least one processor causes the at least one processor to:
receive audio content that includes a voice sample of a voice of a user saying at least one word, the at least one word including a plurality of characters;
analyze the voice sample to identify a sound type in the voice sample, wherein the sound type corresponds to a pronunciation by the user in the voice sample of at least one specified character of the plurality of characters;
generate a filtered voice sample using a personalized filter at least in part by filtering the voice sample to modify the sound type, wherein the personalized filter is customized to the voice of the user based on at least one additional voice sample of the voice of the user; and
output the filtered voice sample.

2. The apparatus of claim 1 , wherein the sound type includes a sibilance.

3. The apparatus of claim 1 , wherein the sound type corresponds to at least a voice type in the pronunciation by the user in the voice sample of the at least one specified character.

4. The apparatus of claim 3 , wherein the voice type corresponds to at least one of a gender or a sex.

5. The apparatus of claim 3 , wherein the voice type corresponds to at least one of an age, an accent, a dialect, or an ethnic background.

6. The apparatus of claim 1 , wherein the sound type corresponds to at least a voice frequency in the pronunciation by the user in the voice sample of the at least one specified character.

7. The apparatus of claim 1 , wherein the sound type corresponds to at least a relative position between a microphone and a person in the pronunciation by the user in the voice sample of the at least one specified character during recording of the audio content using the microphone, wherein the voice sample is spoken by the person.

8. The apparatus of claim 1 , wherein the sound type corresponds to a speech dysfluency, wherein filtering the voice sample includes correcting the speech dysfluency.

9. The apparatus of claim 1 , wherein filtering the voice sample includes applying a filter to the voice sample, wherein the filter includes a de-esser.

10. The apparatus of claim 1 , wherein filtering the voice sample includes applying a filter to the voice sample, wherein the filter includes a compressor that filters a specified frequency range, wherein the audio content includes audio in the specified frequency range, wherein the specified frequency range corresponds to the sound type.

11. The apparatus of claim 1 , wherein filtering the voice sample includes applying a filter to the voice sample, wherein the filter is customized to a voice type corresponding to the voice sample.

12. The apparatus of claim 11 , wherein the filter includes a trained machine learning model that is customized to the voice type, the trained machine learning model having been trained using training data that includes one or more additional voice samples associated with the voice type.

13. The apparatus of claim 12 , wherein the execution of the instructions by the at least one processor causes the at least one processor to:
request at least one of the one or more additional voice samples from a user device, wherein the user device is associated with the voice type.

14. The apparatus of claim 12 , wherein the execution of the instructions by the at least one processor causes the at least one processor to:
update the trained machine learning model using additional training data, wherein the additional training data includes at least the voice sample and the filtered voice sample.

15. The apparatus of claim 1 , wherein outputting the filtered voice sample includes causing the filtered voice sample to be output using an audio output device.

16. The apparatus of claim 1 , wherein outputting the filtered voice sample includes transmitting the filtered voice sample to a recipient device over a communication interface.

17. The apparatus of claim 1 , wherein the apparatus includes a digital signal processor (DSP).

18. A method of audio processing, the method comprising:
receiving audio content that includes a voice sample of a voice of a user saying at least one word, the at least one word including a plurality of characters;
analyzing the voice sample to identify a sound type in the voice sample, wherein the sound type corresponds to a pronunciation by the user in the voice sample of at least one specified character of the plurality of characters;
generating a filtered voice sample using a personalized filter at least in part by filtering the voice sample to modify the sound type, wherein the personalized filter is customized to the voice of the user based on at least one additional voice sample of the voice of the user; and
outputting the filtered voice sample.

19. The method of claim 18 , wherein filtering the voice sample includes applying a filter to the voice sample, wherein the filter is customized to a voice type corresponding to the voice sample.

20. A non-transitory computer readable storage medium having embodied thereon a program, wherein the program is executable by a processor to perform a method of audio processing, the method comprising:
receiving audio content that includes a voice sample of a voice of a user saying at least one word, the at least one word including a plurality of characters;
analyzing the voice sample to identify a sound type in the voice sample, wherein the sound type corresponds to a pronunciation by the user in the voice sample of at least one specified character of the plurality of characters;
generating a filtered voice sample using a personalized filter at least in part by filtering the voice sample to modify the sound type, wherein the personalized filter is customized to the voice of the user based on at least one additional voice sample of the voice of the user; and
outputting the filtered voice sample.

Assignees

• Sony Interactive Entertainment LLC
• Sony Interactive Entertainment Inc

Inventors

• Zhang Jin
• Bean Celeste
• Karimi Sepideh
• Krishnamurthy Sudha

Classifications

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L25/90

  Pitch determination of speech signals · CPC title

• G10L25/51

  for comparison or discrimination · CPC title

• G10L15/187

  Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams · CPC title

• G10L25/60

  for measuring the quality of voice signals · CPC title