Multiple sensory channel approach for translating human emotions in a computing environment

Patent metadata

Field	Value
Publication number	US-9329758-B2
Application number	US-201314071148-A
Country	US
Kind code	B2
Filing date	Nov 4, 2013
Priority date	Aug 13, 2009
Publication date	May 3, 2016
Grant date	May 3, 2016

Abstract

Sensory inputs of a user can be received by a computing device. At least one of the sensory inputs can include a physiological input providing a physiological measurement from a body of the user. Each sensory input can be processed in a unique one of a set of standards-defined sensory channels, each corresponding to a specific emotion dimension. Processing the sensory inputs can transform the physiological measurement into an emotion dimension value. The emotion dimension values from each of the sensory channels can be aggregated to generate at least one emotion datum value, which is a standards-defined value for an emotional characteristic of the user. Historical data for a user can be optionally collected and used by a learning and calibration component to improve the accuracy of the generated emotion datum values for a specific individual. A programmatic action driven by the emotion datum value can be performed.

First claim

What is claimed is:

1. A system for incorporating human emotions in a computing environment comprising:
one or more processors, and a memory storing computer readable program code;
a plurality of discrete sensory channels each for handling sensory input detected via one or more sensors, each of the one or more sensors comprising hardware, wherein different senses are associated with different ones of the discrete sensory channels, wherein each of the discrete sensory channels is a sensory channel corresponding to a specific emotion dimension, wherein sensory input handled within the sensory channels comprises physiological input providing a physiological measurement from a body of the user;
a plurality of in-channel processors, each being one of the one or more processors, that process sensory input specific to the channel and that generate emotion dimension values from the sensory input, wherein each emotion dimension value has been transformed to be independent of idiosyncrasies of a sensory capture device from which the sensory input was originally obtained; and
a sensory aggregator, implemented in the computer readable program code stored in the memory and executable by at least one of the one or more processors, for aggregating emotion dimension values generated in a per-channel basis by the in-channel processors to generate at least one emotion datum value, which is a value for an emotional characteristic of a user from whom the sensory input was gathered, and is an application independent value that is able to be utilized by a plurality of independent applications to discern emotions of said user and to cause application specific code of the independent applications to be reactive to changes in sensory aggregator generated emotion datum values.

2. The system of claim 1 , further comprising:
a plurality of sensory capture devices, wherein said sensory capture devices comprise a physiological sensor, an audio or video capture device, and a user input peripheral for manual user input, wherein input obtained from the physiological sensor, from the audio or video capture device, and from the user input peripheral is handled by separately by different ones of the discrete sensory channels, and wherein said physiological sensor obtains a measure from a system of the human body selected from a group of systems consisting of a cardiovascular system, a muscular system, a circulatory system, a nervous system, a respiratory system, an excretory system, an endocrine system, and a digestive system.

3. The system of claim 1 , further comprising:
a plurality of emotion data consumers each performing a programmatic actions driven by emotion datum values generated by the sensory aggregator.

4. The system of claim 3 , wherein at least one of the emotion data consumers comprises a virtual world software application in which a plurality of humans interact though user specific avatars, wherein said user specific avatars express emotions corresponding to emotions of their users driven by the emotion data values.

5. The system of claim 1 , wherein at least a portion of the in-channel processors are computer program products hosted by a physical computing device remotely located and connected by network to another physical computing device from which the sensory input handled by the portion of the in-channel processors originated.

6. The system of claim 1 , wherein at least one of the in-channel processors analyzes speech input to determine emotions of the user who provided the speech input based upon vocal characteristics of the speech input, and wherein at least one of the in-channel processors analyzes images to determine facial expressions of the user.

7. The system of claim 1 , wherein said in-channel processors and said sensory aggregator are components of a service oriented architecture (SOA), where software services are provided for processing said sensory input and for aggregating said emotion dimension values.

8. A system comprising:
one or more processors;
one or more non-transitory storage mediums storing program instructions, said one or more processors executing the program instructions to:
receive a plurality of sensory inputs of a user;
process each sensory input in a unique one of a plurality of sensory channels, each sensory channel corresponding to a specific emotion dimension, wherein said processing comprises transforming the physiological measurement into an emotion dimension value, said emotion dimension value;

aggregate the emotion dimension values from each of the sensory channels to generate at least one emotion datum value, which is a value for an emotional characteristic of said user, wherein said emotion datum value is an application independent value that is utilized by a plurality of independent applications to discern emotions of said user and to cause application specific code of the independent applications to be reactive to changes in sensory aggregator generated emotion datum values when aggregating the emotion dimension values, adjust user specific factors specified within a stored user specific data set to normalize the generated emotion datum value to a baseline of an average person; and
provide the emotion datum value to the independent applications, wherein each of the independent applications perform a programmatic action driven by the emotion datum value.

9. The system of claim 8 , wherein said user is interacting with a graphical user interface to participate within a virtual environment in which said user is represented in said virtual environment as an avatar, which is presented to a plurality of other users of said virtual environment, wherein a presentation characteristic of said avatar is adjusted for the emotion datum value so that an emotion corresponding to the emotion datum value is presented to the other users.

10. The system of claim 9 , wherein the presentation characteristic alters at least one of a facial expression of the avatar and a speech characteristic of the avatar in accordance with the emotion corresponding to the emotion datum value.

11. The system of claim 8 , wherein the emotion datum value is generated without the user ever explicitly manually inputting semantic content for an emotion.

12. The system of claim 8 , wherein a plurality of independent applications are able to consume the emotion datum value to drive application specific changes based upon emotions of the user.

13. The system of claim 8 , wherein said user participates in an application session of an application which performs said programmatic action driven by the emotion datum value, wherein during the application session, the plurality of sensory inputs from the user are continuously received, processed, and aggregated to continuously generate different emotion datum values, which drive programmatic actions of the application that are based upon emotional variances of the user during the application session, wherein the continuous receiving, processing, aggregating, and performing of the programmatic actions occur in at least one of real-time and near real time, wherein the application session is a communication session in which said user communicates in real-time or near real-time with at least one other user, wherein during said communication session said other user is continuously appraised of emotional changes of said user determined from the continuously generated emotion datum values.

14. The system of claim 8 , wherein processing for different ones of the plurality of channels is handled by different networked computing devices, each of the different networked computing devices being explicitly dedicated to handle at

Assignees

• Guzak Barbara S
• Kwan Hung-Tack
• Vora Janki Y
• IBM

Inventors

• Guzak Barbara S
• Kwan Hung-Tack
• Vora Janki Y

Classifications

• A61B5/6804

  Garments; Clothes · CPC title

• A61B5/16 Primary

  Devices for psychotechnics (using teaching or educational appliances G09B1/00 - G09B7/00); Testing reaction times {; Devices for evaluating the psychological state} · CPC title

• A61B5/0531

  Measuring skin impedance · CPC title

• A61B5/01

  Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20) · CPC title

• G06F3/0484 Primary

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title