Action-based models to identify learned tasks

Patent metadata

Field	Value
Publication number	US-9384448-B2
Application number	US-201113339297-A
Country	US
Kind code	B2
Filing date	Dec 28, 2011
Priority date	Dec 28, 2011
Publication date	Jul 5, 2016
Grant date	Jul 5, 2016

Abstract

Systems provided herein include a learning environment and an agent. The learning environment includes an avatar and an object. A state signal corresponding to a state of the learning environment includes a location and orientation of the avatar and the object. The agent is adapted to receive the state signal, to issue an action capable of generating at least one change in the state of the learning environment, to produce a set of observations relevant to a task, to hypothesize a set of action models configured to explain the observations, and to vet the set of action models to identify a learned model for the task.

First claim

The invention claimed is:

1. A system, comprising:
a three dimensional learning environment comprising an avatar and an object, and a state signal corresponding to a state of the learning environment comprising a location and orientation of the avatar and the object, wherein the object is a camera that produces a set of synthetic images of the three dimensional learning environment and wherein the avatar comprises articulated body parts, such that a processor computes and defines coordinates of each body part of the avatar;
an agent that receives the state signal via the set of synthetic images and issues adjustment instructions that employ depth perception to generate at least one change in the state of the learning environment;
an oracle that generates a reinforcement signal in response to the change in the state of the learning environment and communicates the reinforcement signal to the agent by considering a computer script that orchestrates changes in body-part to body-part and body-part to object spatial relationships with respect to gravity, inertia, and physical occupancy, wherein the agent utilizes the reinforcement signal to produce a set of observations relevant to a task, hypothesizes a set of Internal Action Models configured to explain the observations, and vets the set of Internal Action Models to identify a learned model for the task such that a sum of the reinforcement signals received from the oracle selects the Internal Action Model for the task;
wherein the agent builds a set of input image sequences with increasing reinforcement signals that are indicative of the task; and wherein the agent executes the task autonomously to perform a variety of goal-oriented actions based on real-world imagery.

2. The system of claim 1 , wherein the avatar comprises a plurality of ellipsoids coupled together by a plurality of joint angles.

3. The system of claim 2 , wherein the action issued by the agent comprises a signal corresponding to one of the plurality of joint angles and a negative or positive increment for the one joint angle.

4. The system of claim 1 , wherein hypothesizing the set of Internal Action Models comprises implementing a sequence of springs model.

5. The system of claim 1 , wherein the agent utilizes the learned model to recognize the task when performed by an external agent.

6. The system of claim 1 , wherein vetting the set of Internal Action Models comprises initializing the learning environment to a starting state, attempting a plurality of possible actions, identifying an action of the plurality of actions corresponding to a positive response from an identified Internal Action Model of the set of Internal Action Models, and recording reinforcement signals from the oracle corresponding to each action.

7. The system of claim 1 , comprises one or more virtual camera generates one or more images of the avatar received by the agent as part of the state signal.

8. The system of claim 1 , wherein the agent reduces the set of hypothesized Internal Action Models by computing a consistency function and vets the reduced set of hypothesized Internal Action Models.

Assignees

• Tu Peter Henry
• Yu Ting
• Gao Dashan
• Sebastian Thomas Baby
• Yao Yi
• Gen Electric

Inventors

• Tu Peter Henry
• Yu Ting
• Gao Dashan
• Sebastian Thomas Baby
• Yao Yi

Classifications

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06K9/6256

  Physics · mapped topic

• G06N99/005 Primary

  Physics · mapped topic

• G06N20/00 Primary

  Machine learning · CPC title