Authentication based on heartbeat detection and facial recognition in video data

Patent metadata

Field	Value
Publication number	US-9953231-B1
Application number	US-201615354529-A
Country	US
Kind code	B1
Filing date	Nov 17, 2016
Priority date	Nov 17, 2015
Publication date	Apr 24, 2018
Grant date	Apr 24, 2018

Abstract

Techniques are described for authenticating a user to access information through an application executing on a computing device. Multiple authentication methods may be used in combination to authenticate the user with greater confidence than authentication provided by a single method, and to verify that the user is a live person and not an image or video. Facial recognition may be used in conjunction with heartbeat detection, via video analysis, to verify the user's identity and confirm that the user is live. Facial recognition may include capturing an image of the user's face and comparing certain points on the face with previously gathered information regarding the user. Heartbeat detection may include capturing a video segment of the user, stabilizing the captured video data, applying motion magnification techniques to the stabilized video data, and analyzing the stabilized, magnified video data to determine a presence of the user's heartbeat.

First claim

What is claimed is:

1. A computer-implemented method performed by at least one processor, the method comprising:
receiving, by the at least one processor, video data of a user, the video data collected in response to detecting an attempt to access information through an application executing on a user device, the video data generated by a video capture component of a user device;
comparing, by the at least one processor, at least one frame of the video data to a previously collected image of the user;
analyzing, by the at least one processor, the video data to detect a heartbeat of the user;
providing, by the at least one processor, access to high-risk information through the application based at least partly on:
detecting the heartbeat of the user based on the analyzing of the video data; and
determining that the at least one frame of the video data corresponds to the previously collected image of the user; and

providing, by the at least one processor, access to low-risk information through the application based at least partly on:
failing to detect the heartbeat of the user based on the analyzing of the video data; and
determining that the at least one frame of the video data corresponds to the previously collected image of the user.

2. The method of claim 1 , wherein analyzing the video data to detect the heartbeat further comprises:
performing motion stabilization on the video data; and
magnifying the video data.

3. The method of claim 1 , further comprising:
presenting, by the at least one processor, at least one word through a user interface of the application;
receiving, by the at least one processor, audio data of the user speaking the at least one word, the audio data generated by an audio capture component of the user device; and
comparing, by the at least one processor, the audio data to a stored voice print associated with the user;
wherein providing access to the information through the application is further based on determining that the audio data at least partly corresponds to the stored voice print associated with the user.

4. The method of claim 1 , further comprising:
analyzing, by the at least one processor, the video data to determine a current emotional state of the user;
accessing, by the at least one processor, emotional state data indicating at least one previous emotional state of the user during at least one previous attempt to access information through the application; and
comparing, by the at least one processor, the current emotional state to the at least one previous emotion state;
wherein providing access to the information through the application is further based on determining that the current emotional state at least partly corresponds to the at least one previous emotional state.

5. The method of claim 1 , wherein providing access to the information through the application is further based on one or more of:
determining that an eye color of the user in at least one frame of the video data corresponds to previously determined eye color information for the user;
determining that a personal identification number (PIN) entered by the user corresponds to a valid PIN for the user;
determining that an answer provided by the user in response to a knowledge-based authentication (KBA) question corresponds to a stored answer associated with the user;
determining that an identifier of the user device corresponds to a stored device identifier associated with the user; or
detecting that the user has entered, through a user interface of the application, a one-time passcode previously communicated to the user.

6. The method of claim 1 , further comprising:
determining, by the at least one processor, a score as a sum of weights corresponding to:
detecting the heartbeat of the user; and
determining that the at least one frame of the video data corresponds to the previously collected image of the user;

wherein providing access to the information through the application is further based on determining that the score is at least a threshold score.

7. The method of claim 6 , wherein the threshold score is based at least partly on a risk level of the information to be accessed through the application.

8. The method of claim 6 , wherein the score is determined as the sum of weights further corresponding to one or more of:
determining that an eye color of the user in at least one frame of the video data corresponds to previously determined eye color information for the user;
determining that a personal identification number (PIN) entered by the user corresponds to a valid PIN for the user;
determining that an answer provided by the user in response to a knowledge-based authentication (KBA) question corresponds to a stored answer associated with the user;
determining that an identifier of the user device corresponds to a stored device identifier associated with the user;
detecting that the user has entered, through a user interface of the application, a one-time passcode previously communicated to the user; or
determining that recorded audio data of the user speaking at least partly corresponds to the stored voice print associated with the user.

9. A system comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor, the memory storing instructions which, when executed, cause the at least one processor to perform operations comprising:
receiving video data of a user, the video data collected in response to detecting an attempt to access information through an application executing on a user device, the video data generated by a video capture component of a user device;
comparing at least one frame of the video data to a previously collected image of the user;
analyzing the video data to detect a heartbeat of the user;
providing access to high-risk information through the application based at least partly on:
detecting the heartbeat of the user based on the analyzing of the video data; and
determining that the at least one frame of the video data corresponds to the previously collected image of the user; and

providing access to low-risk information through the application based at least partly on:
failing to detect the heartbeat of the user based on the analyzing of the video data; and
determining that the at least one frame of the video data corresponds to the previously collected image of the user.

10. The system of claim 9 , wherein analyzing the video data to detect the heartbeat further comprises:
performing motion stabilization on the video data; and
magnifying the video data.

11. The system of claim 9 , the operations further comprising:
presenting at least one word through a user interface of the application;
receiving audio data of the user speaking the at least one word, the audio data generated by an audio capture component of the user device; and
comparing the audio data to a stored voice print associated with the user;
wherein providing access to the information through the application is further based on determining that the audio data at least partly corresponds to the stored voice print associated with the user.

12. The system of claim 9 , the operations further comprising:
analyzing the video data to determine a current emotional state of the user;
accessing emotional state data indicating at least one previous emotional state of the user during at least one previous attempt to access information through the application; and
comparing the current emotional state to the at least one previous emotion state;
wherein providing access to the information through the application is further based on determining that the curre

Assignees

• Usaa

Inventors

• Medina Iii Reynaldo
• Chavez Carlos
• Sridaran Nandhini
• Colwell Russell
• Liu Yuesheng
• Briscoe Storme Shelton

Classifications

• G06F21/32 Primary

  using biometric data, e.g. fingerprints, iris scans or voiceprints · CPC title

• G06K9/00892 Primary

  Physics · mapped topic

• G06K9/00617

  Physics · mapped topic

• G06K9/00711

  Physics · mapped topic

• G06K2009/00939

  Physics · mapped topic