Computer systems and methods for securing access to content provided by virtual assistants

Patent metadata

Field	Value
Publication number	US-10650824-B1
Application number	US-201916409261-A
Country	US
Kind code	B1
Filing date	May 10, 2019
Priority date	May 10, 2019
Publication date	May 12, 2020
Grant date	May 12, 2020

Abstract

A computerized method of authenticating a user includes receiving, by a computing device, first biometric data from the user via a first biometric input device; extracting one or more first features from the first biometric data; generating a first vector based on the first features; computing a first similarity score reflecting a geometric distance between the first vector and a first reference vector stored in memory; receiving second biometric data from the user via a second biometric input device; extracting one or more second features from the second biometric data; generating a second vector based on the second features; computing a second similarity score reflecting a geometric distance between the second vector and a second reference vector stored in memory; computing a confidence score based on the first and second similarity scores; and determining a right of the user to access restricted information based on the confidence score.

First claim

What is claimed is:

1. A computerized method of authenticating a user of a virtual assistant operating on a computing device, for each request made by the user during a computing session, the method comprising:
receiving, by the computing device, first biometric data from the user via a first biometric input device in electronic communication with the computing device;
extracting, by the computing device, one or more first features from the first biometric data;
generating, by the computing device, a first vector based on the first features;
computing, by the computing device, using a first neural network in electronic communication with the computing device, a first similarity score reflecting a geometric distance between the first vector and a first reference vector stored in memory of the computing device;
receiving, by the computing device, second biometric data from the user via a second biometric input device in electronic communication with the computing device;
extracting, by the computing device, one or more second features from the second biometric data;
generating, by the computing device, a second vector based on the second features;
computing, by the computing device, using a second neural network in electronic communication with the computing device, a second similarity score reflecting a geometric distance between the second vector and a second reference vector stored in memory of the computing device;
computing, by the computing device, using a third neural network in electronic communication with the computing device, a confidence score based on the first and second similarity scores; and
determining, by the computing device, a right of the user to access restricted information based on the confidence score.

2. The method of claim 1 wherein the first biometric input device is a microphone configured to receive voice input from the user.

3. The method of claim 1 wherein the second biometric input device is a camera configured to receive facial image input from the user.

4. The method of claim 1 further including: monitoring, by the computing device, a stress indication of the user based on the first biometric data at one or more points in time; and denying, by the computing device, access to the restricted information if the stress indication exceeds a specified stress threshold.

5. The method of claim 4 wherein the monitoring occurs periodically or continuously throughout a user interaction.

6. The method of claim 1 further including monitoring, by the computing device, an emotional indication of the user based on the second biometric data at one or more points in time; and denying, by the computing device, access to the restricted information if the emotional indication exceeds a specified emotion threshold.

7. The method of claim 6 wherein the monitoring occurs periodically or continuously throughout user interaction.

8. The method of claim 1 wherein computing at least one of the first similarity score or the second similarity score includes using a supervised learning model.

9. The method of claim 1 wherein the first and second reference vectors are stored in a user profile, the user profile included in an identity manager module of the computing device that interfaces with the virtual assistant, the virtual assistant receiving a request from the user and a voice application for serving the request after the user is authenticated.

10. The method of claim 1 wherein during the computing session the computing device repeats the above steps, continuously or periodically, for newly received first and second biometric data of the user.

11. A computing system for authenticating a user of a virtual assistant operating on a computing device, for each request made by the user during a computing session, the computing system comprising:
the computing device having a processor and a memory;
a first biometric input device in electronic communication with the computing device; and
a second biometric input device in electronic communication with the computing device;
wherein the computing device is configured to: receive first biometric data from the user via the first biometric input device; extract one or more first features from the first biometric data; generate a first vector based on the first features; compute, using a first neural network in electronic communication with the computing device, a first similarity score reflecting a geometric distance between the first vector and a first reference vector stored in the memory of the computing device; receive second biometric data from the user via the second biometric input device; extract one or more second features from the second biometric data; generate a second vector based on the second features; compute, using a second neural network in electronic communication with the computing device, a second similarity score reflecting a geometric distance between the second vector and a second reference vector stored in memory of the computing device; compute, using a third neural network in electronic communication with the computing device, a confidence score based on the first and second similarity scores; and determine a right of the user to access restricted information based on the confidence score.

12. The computing system of claim 11 wherein the first biometric input device is a microphone and the second biometric input device is a camera.

13. The computing system of claim 11 wherein the computing device is one of a smartphone, tablet, or personal computer.

14. The computing system of claim 11 wherein during the computing session the computing device is configured to repeat the above steps, continuously or periodically, for newly received first and second biometric data of the user.

15. A computerized method of training a computing device to recognize a user of a virtual assistant operating on the computing device, for each request made by the user during a computing session, the method comprising:
generating, by the computing device, a device profile for the user, the device profile stored in memory of the computing device and including a device identifier and a user profile identifier;
receiving; by a computing device, first reference biometric data from the user via a first biometric input device in electronic communication with the computing device;
extracting, by the computing device, one or more first reference features from the first reference biometric data;
generating; by the computing device; a first reference vector based on the first reference features;
storing, by the computing device, the first reference vector in a memory section of the computing device associated with the device profile, the first reference vector usable for computation, using a first neural network, of a first similarity score between the first reference vector and a first vector generated based on first biometric data obtained from the user during a computing session;
receiving, by the computing device, second reference biometric data from the user via a second biometric input device in electronic communication with the computing device;
extracting, by the computing device, one or more second reference features from the second reference biometric data;
generating, by the computing device, a second reference vector based on the second reference features; and
storing, by the computing device, the second reference vector in a memory section of the computing device associated with the device profile, the second reference vector usable for computation, using a second neural network, of a second similarity score between the second reference vector a

Assignees

• Fmr Llc

Inventors

• Kesharaju Shanthan
• Mohanty Pranab

Classifications

• G10L2015/225

  Feedback of the input speech · CPC title

• G10L17/18

  Artificial neural networks; Connectionist approaches · CPC title

• G06F21/32

  using biometric data, e.g. fingerprints, iris scans or voiceprints · CPC title

• G10L17/22

  Interactive procedures; Man-machine interfaces · CPC title

• G10L17/00 Primary

  Speaker identification or verification techniques · CPC title