Neural Networks For Speaker Verification

Patent metadata

Field	Value
Publication number	US-2021366491-A1
Application number	US-202117444384-A
Country	US
Kind code	A1
Filing date	Aug 3, 2021
Priority date	Sep 4, 2015
Publication date	Nov 25, 2021
Grant date	—

Abstract

This document generally describes systems, methods, devices, and other techniques related to speaker verification, including (i) training a neural network for a speaker verification model, (ii) enrolling users at a client device, and (iii) verifying identities of users based on characteristics of the users' voices. Some implementations include a computer-implemented method. The method can include receiving, at a computing device, data that characterizes an utterance of a user of the computing device. A speaker representation can be generated, at the computing device, for the utterance using a neural network on the computing device. The neural network can be trained based on a plurality of training samples that each: (i) include data that characterizes a first utterance and data that characterizes one or more second utterances, and (ii) are labeled as a matching speakers sample or a non-matching speakers sample.

First claim

What is claimed is:

1 . A computer-implemented method when executed on data processing hardware of a user device causes the data processing hardware to perform operations comprising:
prompting a user of the user device to speak a plurality of enrollment utterances; receiving audio signals that represent the user speaking the plurality of enrollment utterances, the audio signals representing the user speaking the plurality of enrollment utterances recorded by the user device; generating, using a trained neural network comprising a long short-term memory (LSTM) layer configured to receive the audio signals representing the user speaking the plurality of enrollment utterances as input, a reference speaker model associated with the user that characterizes distinctive features of a voice of the user; storing the reference speaker model on memory hardware of the user device; obtaining a plurality of audio frames representing a first utterance; generating, using the trained neural network, a speaker representation for the utterance, the speaker representation indicating distinctive features of a speaker of the first utterance; determining whether a similarity score between the speaker representation for the first utterance and the reference speaker model stored on the memory hardware of the user device satisfies a similarity score threshold; and when the similarity score satisfies the similarity score threshold, authenticating the speaker of the first utterance as the user associated with the reference speaker model.

2 . The computer-implemented method of claim 1 , wherein the trained neural network further comprises a fully-connected linear layer configured to:
receive, as input, an output of the LSTM layer; and generate, as output, the speaker representation for the first utterance.

3 . The computer-implemented method of claim 1 , wherein the operations further comprise, when the similarity score satisfies the similarity score threshold, updating the reference speaker model associated with the user of the user device based on the first utterance.

4 . The computer-implemented method of claim 1 , wherein the operations further comprise, in response to authenticating the speaker of the first utterance as the user associated with the reference speaker model, transitioning operation of the user device from a low-power state to a more fully-featured state.

5 . The computer-implemented method of claim 1 , wherein the operations further comprise, in response to authenticating the speaker of the first utterance as the user associated with the reference speaker model:
processing one or more terms in the first utterance; and performing an action based on the one or more terms in the first utterance.

6 . The computer-implemented method of claim 1 , wherein the first utterance and each of the plurality of enrollment utterances comprise a same pre-determined phrase.

7 . The computer-implemented method of claim 1 , wherein the similarity score between the speaker representation and the reference speaker model comprises a cosine distance between a vector of values for the speaker representation and a vector of values for the reference speaker model.

8 . The computer-implemented method of claim 1 , wherein the trained neural network is stored on the memory hardware of the user device.

9 . The computer-implemented method of claim 1 , wherein obtaining the plurality of audio frames characterizing the first utterance comprises:
receiving a raw audio signal of the first utterance; segmenting the raw audio signal of the first utterance into a plurality of raw audio frames, each raw audio frame comprising a respective portion of the raw audio signal; and converting the respective portion of the raw audio signal of each raw audio raw audio frame into respective audio features characterizing a respective segment of the first utterance.

10 . The computer-implemented method of claim 1 , wherein the operations further comprise, prior to generating the speaker representation for the first utterance using the trained neural network, receiving the trained neural network over a network from a remote computing device.

11 . A system comprising:
data processing hardware of a user device; and memory hardware of the user device and in communication with the data processing hardware, the memory hardware storing instructions that when executed by the data processing hardware cause the data processing hardware to perform operations comprising:
prompting a user of the user device to speak a plurality of enrollment utterances;
receiving audio signals that represent the user speaking the plurality of enrollment utterances, the audio signals representing the user speaking the plurality of enrollment utterances recorded by the user device;
generating, using a trained neural network comprising a long short-term memory (LSTM) layer configured to receive the audio signals representing the user speaking the plurality of enrollment utterances as input, a reference speaker model associated with the user that characterizes distinctive features of a voice of the user;
storing the reference speaker model on memory hardware of the user device;
obtaining a plurality of audio frames representing a first utterance;
generating, using the trained neural network, a speaker representation for the utterance, the speaker representation indicating distinctive features of a speaker of the first utterance;
determining whether a similarity score between the speaker representation for the first utterance and the reference speaker model stored on the memory hardware of the user device satisfies a similarity score threshold; and
when the similarity score satisfies the similarity score threshold, authenticating the speaker of the first utterance as the user associated with the reference speaker model.

12 . The system of claim 11 , wherein the trained neural network further comprises a fully-connected linear layer configured to:
receive, as input, an output of the LSTM layer; and generate, as output, the speaker representation for the first utterance.

13 . The system of claim 11 , wherein the operations further comprise, when the similarity score satisfies the similarity score threshold, updating the reference speaker model associated with the user of the user device based on the first utterance.

14 . The system of claim 11 , wherein the operations further comprise, in response to authenticating the speaker of the first utterance as the user associated with the reference speaker model, transitioning operation of the user device from a low-power state to a more fully-featured state.

15 . The system of claim 11 , wherein the operations further comprise, in response to authenticating the speaker of the first utterance as the user associated with the reference speaker model:
processing one or more terms in the first utterance; and performing an action based on the one or more terms in the first utterance.

16 . The system of claim 11 , wherein the first utterance and each of the plurality of enrollment utterances comprise a same pre-determined phrase.

17 . The system of claim 11 , wherein the similarity score between the speaker representation and the reference speaker model comprises a cosine distance between a vector of values for the speaker representation and a vector of values for the reference speaker model.

18 . The system of claim 11 , wherein the trained neural network is stored on the memory hardware of the user device.

19 . The system

Assignees

• Google Llc

Inventors

• Heigold Georg
• Bengio Samuel
• Lopez Moreno Ignacio

Classifications

• G10L17/18 Primary

  Artificial neural networks; Connectionist approaches · CPC title

• G10L17/04

  Training, enrolment or model building · CPC title

• G10L17/02 Primary

  Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction · CPC title

• G07C9/37 Primary

  using biometric data, e.g. fingerprints, iris scans or voice recognition · CPC title