Voice morphing apparatus having adjustable parameters

Patent metadata

Field	Value
Publication number	US-11600284-B2
Application number	US-202016740440-A
Country	US
Kind code	B2
Filing date	Jan 11, 2020
Priority date	Jan 11, 2020
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

A voice morphing apparatus having adjustable parameters is described. The disclosed system and method include a voice morphing apparatus that morphs input audio to mask a speaker's identity. Parameter adjustment uses evaluation of an objective function that is based on the input audio and output of the voice morphing apparatus. The voice morphing apparatus includes objectives that are based adversarially on speaker identification and positively on audio fidelity. Thus, the voice morphing apparatus is adjusted to reduce identifiability of speakers while maintaining fidelity of the morphed audio. The voice morphing apparatus may be used as part of an automatic speech recognition system.

First claim

The invention claimed is:

1. A voice morphing apparatus comprising:
a neural network architecture to map input audio data to output audio data, the input audio data comprising a representation of speech from a speaker, the neural network architecture including a set of parameters, the set of parameters being trained to maximize a speaker identification distance from the input audio data to a set of speaker identification vectors and to optimize a speaker intelligibility score for the output audio data.

2. The voice morphing apparatus of claim 1 further comprising a noise filter to pre-process the input audio data.

3. The voice morphing apparatus of claim 2 , wherein the noise filter removes a noise component from the input audio data and the voice morphing apparatus adds the noise component to the set of speaker identification vectors from the neural network architecture.

4. The voice morphing apparatus of claim 1 , wherein the neural network architecture comprises one or more recurrent connections.

5. The voice morphing apparatus of claim 1 , wherein the voice morphing apparatus is configured to output time-series audio waveform data based on the set of speaker identification vectors from the neural network architecture.

6. A non-transitory computer-readable storage medium for storing instructions that, when executed by at least one processor, cause the at least one processor to:
load input audio data from a data source;
input the input audio data to a voice morphing apparatus, the voice morphing apparatus including a set of trainable parameters;
process the input audio data using the voice morphing apparatus to generate morphed audio data;
apply a speaker identification system to at least the morphed audio data to output a measure of speaker identification;
apply an audio fidelity system to the morphed audio data and the input audio data to output a measure of audio fidelity;
evaluate an objective function based on the measure of speaker identification and the measure of audio fidelity; and
adjust the set of trainable parameters for the voice morphing apparatus based on a gradient of the objective function,
wherein the objective function is configured to adjust the set of trainable parameters to optimize the measure of audio fidelity between the morphed audio data and the input audio data and to reduce the measure of speaker identification while maintaining speech intelligibility.

7. A method for optimizing training parameters, the method comprising:
loading input audio data from a data source;
inputting the input audio data to a voice morphing apparatus, the voice morphing apparatus including a set of trainable parameters;
processing the input audio data using the voice morphing apparatus to generate morphed audio data;
applying a speaker identification system to at least the morphed audio data to output a measure of speaker identification;
applying an audio fidelity system to the morphed audio data and the input audio data to output a measure of audio fidelity;
evaluating an objective function based on the measure of speaker identification and the measure of audio fidelity; and
adjusting the set of trainable parameters for the voice morphing apparatus based on a gradient of the objective function,
wherein the objective function is configured to adjust the set of trainable parameters to optimize the measure of audio fidelity between the morphed audio data and the input audio data and to reduce the measure of speaker identification while maintaining speech intelligibility.

Assignees

• Soundhound Inc

Inventors

• Pearson Steve

Classifications

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title