Authenticating a user by correlating speech and corresponding lip shape

Patent metadata

Field	Value
Publication number	US-9754193-B2
Application number	US-201314892011-A
Country	US
Kind code	B2
Filing date	Jun 27, 2013
Priority date	Jun 27, 2013
Publication date	Sep 5, 2017
Grant date	Sep 5, 2017

Abstract

Provided is a method of authenticating a user by correlating speech and corresponding lip shape. An audiovisual of a user requesting authentication is captured. The audiovisual is processed to generate a speech vector quantization sequence and a corresponding lip vector quantization sequence of the user. A likelihood of the speech vector quantization sequence and the corresponding lip vector quantization sequence with probability distributions of speech vector quantization code words corresponding to different lip shape vector quantization code words of the user requesting authentication weighed by probabilities of speech and lip vector quantization indices of the user requesting authentication is evaluated. If upon evaluation, a likelihood of the user requesting authentication being an authentic user is more than a predefined threshold, the user is authenticated.

First claim

The invention claimed is:

1. A method of authentication by correlating speech and corresponding lip shape of a user, comprising:
capturing an audiovisual of a user requesting authentication;
processing the audiovisual to generate a speech vector quantization sequence and a corresponding lip vector quantization sequence of the user;
evaluating a likelihood of the speech vector quantization sequence and the corresponding lip vector quantization sequence to be from an authentic user based on co-occurrence probability distributions of consecutive speech vector quantization code words within a lip frame in the lip vector quantization sequence at a particular time, and weighted by probability distribution of vector quantization indices of speech and lip features corresponding to the user as indexed in a codebook; and
authenticating the user, if upon evaluation, a likelihood of the user requesting authentication being the authentic user is more than a predefined threshold.

2. The method of claim 1 , wherein the likelihood of the user requesting authentication being the authentic user is determined in accordance with the following equation:

Likelihood

Like
N

=

∏

t
=
1

T

⁢

⁢

∏

i
=
1

3

⁢

⁢

P
⁡

(

S

t
⁢

⁢
…
⁢

⁢
i

/

L
t

)

N

⁢

P
⁡

(

S

t
⁢

⁢
…
⁢

⁢
i

/
N

)

⁢

P
⁡

(

L
i

/
N

)

where, “N” is the user requesting authentication, “t” indexes time of the lip vector quantization sequence and “i” indexes three consecutive speech vector quantization code words within each lip frame in the lip vector quantization sequence, P(S i /L i ) N represents co-occurrence probability distribution, and P(S i /N), and P(L i /N) represent probability distributions of vector quantization indices of speech and lip features correspondingly, of the user requesting authentication ‘N’.

3. The method of claim 1 , further comprising generating the probability distributions of speech vector quantization code words corresponding to different lip shapes of the authentic user.

4. The method of claim 3 , wherein generating the probability distributions of speech vector quantization code words corresponding to different lip shapes of the authentic user comprises:
capturing an audiovisual of the authentic user;
processing the audiovisual to generate a speech vector quantization sequence and a corresponding lip vector quantization sequence of the authentic user; and
determining probability distributions of the speech vector quantization code words corresponding to different lip shapes of the authentic user.

5. The method of claim 1 , wherein the speech vector quantization sequence and the corresponding lip vector quantization sequence are compared using a different sampling interval.

6. The method of claim 1 , further comprising generating a universal vector quantization codebook of speech signals for a plurality of users including the authentic user.

7. The method of claim 1 , further comprising generating a universal vector quantization codebook of lip shapes for a plurality of users including the authentic user.

8. The method of claim 1 , further comprising generating the probability distributions of speech vector quantization code words and the probability distributions of lip shape vector quantization code words for each enrolled user.

9. A system, comprising:
a processor; and
a memory coupled to the processor, wherein the memory includes an authentication module to:
process an audiovisual of a user requesting authentication to generate a speech vector quantization sequence and a corresponding lip vector quantization sequence of the user;
evaluate a likelihood of the speech vector quantization sequence and the corresponding lip vector quantization sequence to be from an authentic user based on co-occurrence probability distributions of consecutive speech vector quantization code words within a lip frame in the lip vector quantization sequence at a particular time, and weighted by probability distribution of vector quantization indices of speech and lip features corresponding to the user as indexed in a codebook; and
authenticate the user, if upon evaluation, a likelihood of the user requesting authentication being the authentic user is more than a predefined threshold.

Assignees

• Hewlett Packard Development Co Lp
• Ramachandrula Sitaram
• Ravishankar Hariharan

Inventors

• Ramachandrula Sitaram
• Ravishankar Hariharan

Classifications

• G10L17/10 Primary

  Multimodal systems, i.e. based on the integration of multiple recognition engines or fusion of expert systems · CPC title

• G06V10/806

  of extracted features · CPC title

• G06F18/256

  of results relating to different input data, e.g. multimodal recognition · CPC title

• G06F18/253

  of extracted features · CPC title

• G06K9/629

  Physics · mapped topic