System and method for guiding social interactions

Patent metadata

Field	Value
Publication number	US-2019139438-A1
Application number	US-201715807688-A
Country	US
Kind code	A1
Filing date	Nov 9, 2017
Priority date	Nov 9, 2017
Publication date	May 9, 2019
Grant date	—

Abstract

A social interaction system and of operation thereof for interpreting social cues and using such analysis to improve social interaction via the providing of a suggested response is provided. The social interaction system includes one or more sensors to obtain social indicator data of one or more individuals in an environment during a social interaction, the social indicator data related to a behavior of the one or more individuals. The social interaction system also includes a processing system configured to determine a social state of the one or more individuals using the social indicator data and determine an optimal social response of a person to the social interaction based on an analysis of the social state. The social interaction system further includes a feedback system to indicate the optimal social response to the person.

First claim

What is claimed is:

1 . A social interaction system comprising:
one or more sensors to obtain social indicator data of one or more individuals in an environment during a social interaction, the social indicator data related to a behavior of the one or more individuals; a processing system configured to:
determine a social state of the one or more individuals using the social indicator data; and
determine an optimal social response of a person to the social interaction based on an analysis of the social state; and
a feedback system to indicate the optimal social response to the person.

2 . The social interaction system of claim 1 wherein the processing system uses reinforcement learning to determine the optimal social response, with the processing system configured to:
determine the optimal social response using a policy to map the social state to the optimal social response,
determine whether the optimal social response results in a desirable social outcome of the social interaction; and
update the policy to optimize a future social response based on an analysis of the social outcome.

3 . The social interaction system of claim 1 wherein the processing system uses domain knowledge to determine the optimal social response by mapping the social state to the optimal social response using a policy that has predefined relationships between mapped social states and social responses.

4 . The social interaction system of claim 1 wherein the feedback system operates in real-time.

5 . The social interaction system of claim 4 wherein the feedback system comprises a wearable augmented reality system.

6 . The social interaction system of claim 1 wherein the one or more sensors comprises at least one camera or at least one microphone.

7 . The social interaction system of claim 6 wherein the one or more sensors comprises at least one wearable sensor.

8 . The social interaction system of claim 1 wherein the processing system uses a probabilistic inference engine to determine the social state based on the social indicator data by inferring a social state estimated to result in the behavior of the one or more individuals.

9 . The social interaction system of claim 8 wherein the social state is a group social state determined by the probabilistic inference engine based on group behavioral statistics.

10 . The social interaction system of claim 1 wherein the processing system is a wearable processing device.

11 . A system for assisting a user with social interaction, the system comprising:
one or more sensors to obtain data indicating social expression of one or more individuals; a processing system programmed to:
extract social cues from the social expression data, and
determine a social response based on the social cues to assist the user interact socially with the one or more individuals, the social response determined by applying a predefined policy based upon social outcomes; and
a feedback system to indicate the social response to the user.

12 . The system of claim 11 wherein, in determining the social response, the processing system is programmed to apply the predefined policy using domain knowledge by mapping the social cues to the social response using a predefined relationship between mapped social cues and social responses.

13 . The system of claim 11 wherein in determining the social response, the processing system is programmed to apply the predefined policy using machine learning based upon:
a state space determined from the social cues,
an action space to determine the social response from the state space, and
a reward based on a social outcome of the user interacting socially with the one or more individuals resulting from the social response; and
wherein the predefined policy optimizes the social response by maximizing the reward.

14 . The system of claim 11 wherein the one or more sensors comprises an audio, visual, or physiological sensor.

15 . The system of claim 11 wherein the processing system further comprises an inference engine to determine a social state based on the social cues; and
wherein the predefined policy maps the social state to the social response.

16 . The system of claim 11 wherein the feedback system is a wearable feedback system.

17 . A non-transitory computer readable storage medium having stored thereon a computer program for optimizing social outcomes, the computer program comprising instructions that, when executed by a processor, cause the processor to:
retrieve data of one or more persons involved in a social interaction using one or more sensors; extract social cues from the social interaction using the data; estimate a social state based on the social cues of the one or more persons involved in the social interaction; and map the social state to a suggested action of a person to engage in the social interaction using a policy that optimizes suggested actions based on social outcomes.

18 . The non-transitory computer readable storage medium of claim 17 wherein the policy has a predefined relationship between social states and suggested actions.

19 . The non-transitory computer readable storage medium of claim 17 wherein the policy optimizes future suggested actions based upon previous social outcomes.

20 . The non-transitory computer readable storage medium of claim 17 wherein the data is retrieved from at least one of an audio sensor or visual sensor.

21 . The non-transitory computer readable storage medium of claim 17 wherein the data indicates verbal or non-verbal social expression.

22 . The non-transitory computer readable storage medium of claim 17 wherein the policy optimizes future suggested actions based upon delayed rewards.

23 . The non-transitory computer readable storage medium of claim 17 wherein the instructions further cause the processor to provide the suggested action to a feedback system.

24 . The non-transitory computer readable storage medium of claim 23 wherein the feedback system operates in a forensic feedback mode.

Assignees

• Gen Electric

Inventors

• Tu Peter Henry
• Gao Tao
• Tu Jilin

Classifications

• G06K9/00671

  Physics · mapped topic

• G06N99/005

  Physics · mapped topic

• G09B5/04

  with audible presentation of the material to be studied (sound recording or reproducing G11B) · CPC title

• G09B5/02

  with visual presentation of the material to be studied, e.g. using film strip · CPC title

• G09B19/00 Primary

  Teaching not covered by other main groups of this subclass (teaching or practice apparatus for gun-aiming or gun-laying F41G3/26) · CPC title