Predictive video analytics system and methods

Patent metadata

Field	Value
Publication number	US-2016133274-A1
Application number	US-201614996913-A
Country	US
Kind code	A1
Filing date	Jan 15, 2016
Priority date	Oct 27, 2014
Publication date	May 12, 2016
Grant date	—

Abstract

The methods and systems described herein predict user behavior based on analysis of a user video communication. The methods include receiving a user video communication, extracting video facial analysis data from the video communication, extracting voice analysis data from the video communication, associating the video facial analysis data with the voice analysis data to determine an emotional state of a user, collecting biographical profile information specific to the user, applying a linguistic-based psychological behavioral model to the spoken words to determine personality type of the user, and inputting the collected biographical profile information, emotional state, and personality type into a predictive model to determine a likelihood of an outcome of the video communication.

First claim

What is claimed is:

1 . A video analytics system adapted to predict user behavior based on analysis of a video communication, which comprises:
a node comprising a processor and a non-transitory computer readable medium operably coupled thereto, the non-transitory computer readable medium comprising a plurality of instructions stored in association therewith that are accessible to, and executable by, the processor, where the plurality of instructions comprises:
instructions that, when executed, receive a video communication from a user, wherein the video communication comprises an audio component and a video component;
instructions that, when executed, analyze the video component to provide time-coded video behavioral data from the user;
instructions that, when executed, analyze the audio component to provide time-coded spoken words from the user;
instructions that, when executed, associate the time-coded spoken words with the video behavioral data to determine an emotional state of the user;
instructions that, when executed, collect biographical profile information specific to the user;
instructions that, when executed, determine a personality type of the user by applying a linguistic-based algorithm to the spoken words, searching a density of keywords in the spoken words, and comparing the keywords to a library separated by different personality types; and
instructions that, when executed, enter the collected biographical profile information, the emotional state, and the personality type into a predictive model, wherein the predictive model generates an indication of a likelihood of an outcome of the video communication.

2 . The system of claim 1 , wherein the outcome comprises one or more of whether a user will terminate his or her account, whether a user will purchase a product, whether a user is a fraudster, and whether a user will initiate additional subsequent interaction sessions regarding an issue.

3 . The system of claim 1 , wherein the video component includes a non-verbal, non-textual element comprising one or more of eye movement, facial expressions, gestures, activities, body postures, behaviors, attire, and actions.

4 . The system of claim 1 , wherein more than one user is speaking in the video communication.

5 . The system of claim 4 , wherein the audio and video components are analyzed simultaneously to determine which user is speaking.

6 . The system of claim 1 , which further comprises instructions, that when executed, identify attire of the user.

7 . The system of claim 6 , which further comprises instructions, that when executed, determine user value data based on estimated cost of the attire.

8 . The system of claim 6 , which further comprises instructions, that when executed, recommend items based on the attire of the user and display the items to an agent.

9 . The system of claim 1 , which further comprises instructions, that when executed, determine distress and engagement of the user.

10 . The system of claim 9 , wherein the distress and engagement are entered into the predictive model.

11 . The system of claim 1 , which further comprises instructions, that when executed, build the predictive model using previously-obtained emotional states and personality types of a plurality of different users.

12 . The system of claim 11 , wherein the emotional states are aggregated by personality types.

13 . The system of claim 1 , which further comprises instructions, that when executed:
provide the indication of the outcome to an agent; determine, based at least in part on the indication of the likelihood of the outcome of the video communication, whether the agent should speak specific words, perform one or more specific actions, or both; and provide the agent with the specific words, the one or more specific actions, or both.

14 . The system of claim 1 , which further comprises instructions, that when executed:
determine the likelihood of the outcome of the video communication should be increased or decreased; select a second agent for transferring the video communication to, wherein the transferring of the video communication to the second agent is determined, using at least the predictive model, to increase or decrease the likelihood of the outcome occurring; and transfer the video communication to the second agent.

15 . A method to predict user behavior based on analysis of a video communication, which comprises:
receiving, by one or more processors, a user video communication; extracting, by the one or more processors, video facial analysis data for the user from the video communication; extracting, by the one or more processors, voice analysis data from the user video communication; associating, by the one or more processors, the video facial analysis data with the voice analysis data to determine an emotional state of the user; collecting, by the one or more processors, biographical profile information specific to the user; determining, by the one or more processors, a personality type of the user by applying a linguistic-based algorithm to the spoken words, searching a density of keywords in the spoken words, and comparing the keywords to a library separated by different personality types; and entering, by the one or more processors, the collected biographical profile information, the emotional state, and the personality type into a predictive model, wherein the predictive model generates an indication of a likelihood of an outcome of the video communication.

16 . The method of claim 15 , wherein the outcome comprises one or more of whether a user will terminate his or her account, whether a user will pay an outstanding bill, whether a user is a fraudster, and whether a user will initiate additional subsequent interaction sessions regarding an issue.

17 . The method of claim 15 , which further comprises analyzing the user's clothing and accessories.

18 . The method of claim 17 , which further comprises determining user value data based on the analysis of the user's clothing and accessories.

19 . The method of claim 15 , which further comprises generating time-coded distress and engagement data for the user.

20 . The method of claim 19 , which further comprises aggregating the time-coded distress and engagement data with the video facial analysis data and the voice analysis data.

21 . The method of claim 15 , which further comprises displaying the outcome to an agent and providing the agent with specific words, specific actions, or both.

22 . The method of claim 15 , which further comprises automatically sending an email with a special offer to the user based on the outcome.

23 . The method of claim 15 , which further comprises generating an agent performance score or generating agent training materials based on the outcome.

24 . A non-transitory machine-readable medium comprising instructions which, in response to a computer system, cause the computer system to perform a method which comprises:
receiving a user video communication; separating an audio component from a video component of the video communication; analyzing facial expressions of the user in the video component; transcribing words spoken of the user in the audio component; associating the facial expressions and spoken words to determine an emotional state of the user; collecting biographical profile information specific to the user; determining a personality type of the u

Assignees

• Mattersight Corp

Inventors

• Conway Kelly
• Danson Christopher

Classifications

• G10L17/22

  Interactive procedures; Man-machine interfaces · CPC title

• G10L17/26

  Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices · CPC title

• G06K9/00302

  Physics · mapped topic

• G10L25/63 Primary

  for estimating an emotional state · CPC title

• G06V40/179

  metadata assisted face recognition · CPC title