Emotion detection system

Patent metadata

Field	Value
Publication number	US-9600715-B2
Application number	US-201514752100-A
Country	US
Kind code	B2
Filing date	Jun 26, 2015
Priority date	Jun 26, 2015
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

Emotion detection systems disclosed herein actively protect the privacy and security of monitored subjects. This provision of protection differentiates these systems from conventional emotion detectors. In some examples, a self-contained emotion detection device protects the identity of a subject while providing anonymous data descriptive of the subject's emotional state to other devices within the emotion detection system. In these examples, the emotion detection device protects the identity of the subject by isolating raw image data acquired by the device exclusively within storage locations encapsulated within the device. To provide the anonymous data, the emotion detection device first processes the raw image data to identify landmark points within an image of the subject's face. Next, the emotion detection device normalizes the landmark points to a generic face and transmits an anonymous data set descriptive of the normalized landmark points to the other devices for subsequent processing.

First claim

What is claimed is:

1. An emotion detection system comprising an emotion detection device including:
a memory;
at least one processor coupled to the memory; and
a face image processing module executable by the at least one processor and configured to:
receive image data depicting a face of a subject;
analyze the image data to identify a plurality of points on the face;
store an anonymous data set descriptive of the plurality of points, each element of the anonymous data set including an identifier field and a location field;
transmit the anonymous data set to a remote device distinct from the emotion detection device; and
delete the image data.

2. The emotion detection system of claim 1 , wherein the image data includes at least one of two-dimensional image data and three-dimensional image data.

3. The emotion detection system of claim 1 , wherein the face image processing module is configured to:
analyze the image data at least in part by normalizing a data set to a generic face; and
store the anonymous data at least in part by storing the data set.

4. The emotion detection system of claim 1 , wherein the image data includes landmark points.

5. The emotion detection system of claim 1 , wherein the location field of each element of the anonymous data set includes at least one of a two-dimensional value and a three-dimensional value.

6. The emotion detection system of claim 1 , wherein the emotion detection device further comprises a camera configured to acquire the image data, and wherein the face image processing module is configured to deactivate the camera prior to analyzing the image data, storing the anonymous data set, and transmitting the anonymous data set.

7. The emotion detection system of claim 6 , wherein the emotion detection device further comprises a housing that includes the camera, the memory, the at least one processor, and the face image processing module.

8. The emotion detection system of claim 1 , further comprising the remote device, wherein the remote device is configured to:
identify a stimulus provided to the subject prior to receiving the image data; and
analyze the anonymous data set to determine an emotional state of the subject when presented the stimulus.

9. The emotion detection system of claim 8 , wherein the stimulus includes at least one of non-interactive content and interactive content.

10. The emotion detection system of claim 8 , wherein the stimulus includes a portion of content provided to the subject.

11. The emotion detection system of claim 10 , wherein the portion of content includes a sequence of video frames.

12. The emotion detection system of claim 1 , further comprising a stimulus controller configured to stimulate the subject.

13. The emotion detection system of claim 12 , further comprising the remote device, wherein the remote device is configured to transmit a message to the stimulus controller and the stimulus controller is configured to simulate the subject in response to receiving the message.

14. The emotion detection system of claim 1 , further comprising the remote device, wherein the remote device is configure to:
analyze the anonymous data to identify an estimated emotional state of the subject;
identify a target emotional state for the subject based on the estimated emotional state of the subject;
identify a stimulus to transition the subject to the target emotional state; and
stimulate the subject with the stimulus.

15. A method of detecting emotions using a device, the method comprising:
receiving image data depicting a face of a subject;
analyzing the image data to identify a plurality of points on the face;
storing an anonymous data set descriptive of the plurality of points, each element of the anonymous data set including an identifier field and a location field;
transmitting the anonymous data set to a remote device distinct from the device; and
deleting the image data.

16. The method of claim 15 , wherein analyzing the image data includes normalizing a data set to a generic face and storing the anonymous data set includes storing the data set.

17. The method of claim 15 , wherein analyzing the image data includes identifying landmark points.

18. The method of claim 15 , further comprising:
acquiring the image data via a camera; and
deactivating the camera prior to analyzing the image data, storing the anonymous data set, and transmitting the anonymous data set.

19. The method of claim 15 , further comprising:
identifying a stimulus provided to the subject prior to receiving the image data; and
analyzing the anonymous data set to determine an emotional state of the subject when presented the stimulus.

20. The method of claim 19 , wherein identifying the stimulus includes identifying a portion of content provided to the subject.

21. The method of claim 20 , wherein identifying the portion of content includes identifying at least one of a sequence of video frames within the content and one or more time offsets within the content.

22. A non-transient computer program product encoded with instructions that when executed by one or more processors of a device cause a process for detecting emotions to be carried out, the process comprising:
receiving image data depicting a face of a subject;
analyzing the image data to identify a plurality of points on the face;
storing an anonymous data set descriptive of the plurality of points, each element of the anonymous data set including an identifier field and a location field;
transmitting the anonymous data set to a remote device distinct from the device; and
deleting the image data.

23. The computer program product of claim 22 , wherein analyzing the image data includes normalizing a data set to a generic face and storing the anonymous data set includes storing the data set.

24. The computer program product of claim 22 , further comprising:
identifying a stimulus provided to the subject prior to receiving the image data; and
analyzing the anonymous data set to determine an emotional state of the subject when presented the stimulus.

Assignees

• Intel Corp

Inventors

• Natan Ofir
• Nabwani Ayoob

Classifications

• G06K9/00281

  Physics · mapped topic

• G06K2009/00328

  Physics · mapped topic

• G06K9/00315 Primary

  Physics · mapped topic

• G06K9/42

  Physics · mapped topic

• G06V40/174 Primary

  Facial expression recognition · CPC title