Continual selection of scenarios based on identified tags describing contextual environment of a user for execution by an artificial intelligence model of the user by an autonomous personal companion

Patent metadata

Field	Value
Publication number	US-11568265-B2
Application number	US-201715684830-A
Country	US
Kind code	B2
Filing date	Aug 23, 2017
Priority date	Aug 23, 2017
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

An autonomous personal companion executing a method including capturing data related to user behavior. Patterns of user behavior are identified in the data and classified using predefined patterns associated with corresponding predefined tags to generate a collected set of one or more tags. The collected set is compared to sets of predefined tags of a plurality of scenarios, each to one or more predefined patterns of user behavior and a corresponding set of predefined tags. A weight is assigned to each of the sets of predefined tags, wherein each weight defines a corresponding match quality between the collected set of tags and a corresponding set of predefined tags. The sets of predefined tags are sorted by weight in descending order. A matched scenario is selected for the collected set of tags that is associated with a matched set of predefined tags having a corresponding weight having the highest match quality.

First claim

What is claimed is:

1. A method, comprising:
capturing global data related to behaviors of a plurality of users and a plurality of contextual environments experienced by the plurality of users;
generating a plurality of predefined patterns for the behaviors and the plurality of contextual environments using a deep learning engine processing the global data that are captured and provided as input, wherein the plurality of predefined patterns is associated with a plurality of predefined tags;
capturing local data related to behavior of a user and an environment of the user at an autonomous and robotic personal companion providing services to the user, wherein movement of the autonomous and robotic personal companion when capturing the local data is independent of control from the user;
analyzing at a server the local data to identify patterns of user behavior and a contextual environment of the user from the plurality of predefined patterns, wherein each of the plurality of predefined patterns is associated with a corresponding predefined tag;
classifying at the server the patterns of user behavior and the contextual environment that are identified as a collected set of tags, wherein tags in the collected set are associated with the patterns of user behavior and the contextual environment that are identified;
comparing at the server the collected set of tags to each of a plurality of sets of predefined tags associated with a plurality of scenarios, wherein each scenario corresponds to predefined patterns of user behavior within a corresponding contextual environment, wherein the each scenario includes a corresponding set of predefined tags;
assigning at the server a weight to the each of the plurality of sets of predefined tags based on the comparing the collected set of tags to the each of the plurality of sets of predefined tags, wherein each weight defines a corresponding match quality between the collected set of tags and the corresponding set of predefined tags;
selecting at the server a matched scenario to the collected set of tags, wherein the matched scenario is associated with a matched set of predefined tags having a corresponding weight with a highest match quality;
executing at the server a matched algorithm of the matched scenario with the local data that are captured to generate a result, wherein each of the plurality of scenarios has a corresponding algorithm; and
automatically controlling the autonomous and robotic personal companion to perform an action based on the result for interacting with the user and without selection of the action by the user.

2. The method of claim 1 , further comprising:
providing the local data that are captured as input into the matched algorithm of the matched scenario to determine a behavior associated with the personal companion; and
performing one or more actions based on the behavior that is determined, wherein at least one action includes moving the personal companion.

3. The method of claim 2 , wherein the providing the local data that are captured as the input into the matched algorithm includes:
determining an emotional state of the user based on at least one of the collected set of tags; and
providing a therapy based on the emotional state as one of the one or more actions.

4. The method of claim 2 , wherein the providing the local data that are captured as the input into the matched algorithm further comprises:
determining an emotional state of the user based on at least one of the collected set of tags; and
providing animation of an object reflecting the emotional state as one of the one or more actions.

5. The method of claim 2 , further comprising:
determining when moving that the autonomous and robotic personal companion is approaching a private zone in physical space; and
preventing the personal companion from entering the private zone.

6. The method of claim 2 , further comprising:
positioning the autonomous and robotic personal companion closer to the user when performing the moving.

7. The method of claim 2 , further comprising:
following the user when performing the moving.

8. The method of claim 2 , further comprising:
positioning the autonomous and robotic personal companion when moving to project images from the personal companion onto a displayable surface; and
projecting the images as one of the one or more actions.

9. The method of claim 2 , wherein the matched algorithm selects the one or more actions to be performed from a plurality of possible actions.

10. The method of claim 2 , further comprising:
starting a gaming application for play by the user as one of the one or more actions.

11. The method of claim 1 , wherein the generating the plurality of predefined patterns includes:
accessing data related to monitored behavior of a plurality of users;
determining the plurality of predefined patterns based on the data related to monitored behavior of a plurality of users.

12. The method of claim 1 , further comprising:
collecting the local data on a continual basis;
determining a change of context based on the collected set of tags that are updated;
comparing the collected set of tags that is updated to the each of the plurality of sets of predefined tags associated with the plurality of scenarios;
assigning an updated weight to the each of the plurality of sets of predefined tags based on the comparing;
sorting the plurality of sets of predefined tags by corresponding updated weights in descending order; and
selecting an updated matched scenario to the collected set of tags that is updated and that is associated with an updated matched set of predefined tags having a corresponding updated weight with the highest match quality.

13. The method of claim 1 , further comprising:
setting an expiration period for each of a plurality of algorithms of the plurality of scenarios, wherein execution of a corresponding algorithm given the local data that are captured as input data provides the action to be taken by the autonomous and robotic personal companion.

14. The method of claim 1 , further comprising:
determining audio data from the local data that are captured based on at least one of the collected set of tags;
classifying the audio data into one of command speech, background scenario speech, and conversation speech; and
aligning the matched scenario with the audio data that is classified.

15. A non-transitory computer-readable medium storing a computer program for implementing a method, the computer-readable medium comprising:
program instructions for capturing global data related to behaviors of a plurality of users and a plurality of contextual environments experienced by the plurality of users;
program instructions for generating a plurality of predefined patterns for the behaviors and the plurality of contextual environments using a deep learning engine processing the global data that are captured and provided as input, wherein the plurality of predefined patterns is associated with a plurality of predefined tags;
program instructions for capturing local data related to behavior of a user and an environment of the user at an autonomous and robotic personal companion providing services to the user, wherein movement of the autonomous and robotic personal companion when capturing the local data is independent of control from the user;
program instructions for analyzing at a server the local data to identify patterns of user behavior and a contextual environment of the user from the plurality of predefined patterns, wherein each of the plurality of predefined patterns is associated with a corres

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Taylor Michael
• Fernandez-Rico Javier
• Bashkirov Sergey
• Yoo Jaekwon
• Chen Ruxin

Classifications

• G06N3/086 Primary

  using evolutionary algorithms, e.g. genetic algorithms or genetic programming · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N3/008 Primary

  based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour · CPC title