Information processing apparatus, information processing method, and computer program for controlling state transition

Patent metadata

Field	Value
Publication number	US-RE46186-E
Application number	US-201313927708-A
Country	US
Kind code	E1
Filing date	Jun 26, 2013
Priority date	Mar 13, 2008
Publication date	Oct 25, 2016
Grant date	Oct 25, 2016

Abstract

An information processing apparatus includes: model learning means for self-organizing, on the basis of a state transition model having a state and state transition to be learned by using time series data as data in time series, an internal state from an observation signal obtained by a sensor; and controller learning means for performing learning for allocating a controller, which outputs an action, to each of transitions of a state or each of transition destination states in the state transition model indicating the internal state self-organized by the model learning means.

First claim

What is claimed is:

1. An information processing apparatus comprising:
model learning means for self-organizing, on the basis of a state transition model having a state and state transition to be learned by using time series data as data in time series, an internal state from an observation signal obtained by a sensor;
controller learning means for performing learning for allocating a controller, which outputs an action, to each of transitions of a state or each of transition destination states in the state transition model indicating the internal state self-organized by the model learning means;
initial-structure setting means for initializing structure of the state transition model to sparse structure;
data adjusting means for adjusting the time series data used for the learning according to progress of the learning and outputting time series data after the adjustment;
parameter estimating means for estimating a parameter of the state transition model using the time series data after adjustment; and
structure adjusting means for adjusting the structure of the state transition model.

2. An information processing apparatus according to claim 1 , further comprising:
planning means for planning a path for attaining a target as a transition sequence of a state on the state transition model indicating the internal state self-organized by the model learning means; and
execution managing means for invoking, for each of transitions included in the path planned by the planning means, the controller allocated by the controller learning means to manage execution of an action along the path.

3. An information processing apparatus according to claim 2 , wherein
the model learning means self-organizes, independently for each of plural modals, an internal state from an observation signal obtained by a sensor of a modal corresponding thereto on the basis of state transition models, and
the information processing apparatus further includes causality means for estimating causality of transition in one state transition model and a state of another state transition model among the state transition models for each of the plural modals respectively indicating the internal state self-organized by the model learning means.

4. An information processing apparatus according to claim 3 , wherein the execution managing means causes, when it is difficult to directly control an internal state of a predetermined modal among the plural modals respectively indicating the internal state self-organized by the model learning means, the planning means to recursively execute planning to control the internal state on the basis of the causality estimated by the causality means.

5. An information processing apparatus according to claim 2 , further comprising setting means for spontaneously setting a target from the internal state self-organized by the model learning means, wherein
the controller learning means, the planning means, and the execution managing means execute respective kinds of processing to realize the target spontaneously set by the setting means.

6. An information processing method comprising the steps of:
self-organizing, on the basis of a state transition model having a state and state transition to be learned by using time series data as data in time series, an internal state from an observation signal obtained by a sensor;
performing learning for allocating a controller, which outputs an action, to each of transitions of a state or each of transition destination states in the state transition model indicating the self-organized internal state self-organized;
initializing structure of the state transition model to sparse structure;
adjusting the time series data used for the learning according to progress of the learning and outputting time series data after the adjustment;
estimating a parameter of the state transition model using the time series data after adjustment; and
adjusting the structure of the state transition model.

7. An information processing apparatus according to claim 1 , wherein the data adjusting means adjusts, according to the progress of the learning, the time series data from data including a macro characteristic to data including a micro characteristic.

8. An information processing apparatus according to claim 1 , wherein the structure adjusting means adjusts the structure of the state transition model by performing division of a state of the state transition model, merging of a state, the addition of a state, addition of state transition, deletion of a state, or deletion of state transition.

9. An information processing apparatus according to claim 1 , further comprising evaluating means for evaluating the state transition model for which the learning is performed and determining, on the basis of a result of the evaluation of the state transition model, whether the learning should be finished.

10. An information processing apparatus according to claim 1 , wherein the state transition model is an HMM (Hidden Markov Model).

11. An information processing apparatus according to claim 10 , wherein the structure adjusting means performs deletion of a state with a state not forming a path calculated by a Viterbi method among states of the HMM set as a target.

12. An information processing apparatus according to claim 10 , wherein the structure adjusting means performs deletion of state transition with state transition not forming a path calculated by a Viterbi method among state transitions of the HMM set as a target.

13. An information processing apparatus according to claim 1 , further comprising:
detecting means for detecting an event that occurs immediately preceding state transition that occurs in a first set including events exclusive to one another, the event being an event in a second set, which is a single or plural other sets including events exclusive to one another; and
estimating means estimating, with the state transition set as a result event and the event in the second set detected by the detecting means set as a cause event, causality between the events included in the different sets.

14. An information processing apparatus according to claim 13 , wherein the estimating means calculates, for each of events that occur in the second set immediately preceding the state transition and are detected by the detecting means, a conditional probability concerning the state transition and estimates causality between the events included in the different sets.

15. An information processing apparatus according to claim 14 , wherein
the detecting means detects, for a first event immediately preceding occurrence of the state transition and a second event in the second set that occurs simultaneously with the first event, a first number of times the first and second events simultaneously occur immediately preceding the state transition and a second number of times the first and second events simultaneously occurs, and
the estimating means calculates a conditional probability concerning the state transition by dividing the first number of times detected by the detecting means by the second number of times detected by the detecting means.

16. An information processing apparatus according to claim 14 , further comprising storing means for storing, in association with each other, each of the events that occur in the second set immediately preceding the state transition and the conditional probability concerning the state transition calculated for each of the events by the estimating means.

17. An information processing apparatus according to claim 16 , further compris

Assignees

• Sony Corp

Inventors

• Sabe Kohtaro
• Minamino Katsuki
• Kawamoto Kenta
• Suzuki Hirotaka
• Hidai Kenichi

Classifications

• G06N99/005

  Physics · mapped topic

• G05B2219/23288

  Adaptive states; learning transitions · CPC title

• G05B19/045 Primary

  using logic state machines, consisting only of a memory or a programmable logic device containing the logic for the controlled machine and in which the state of its outputs is dependent on the state of its inputs or part of its own output states, e.g. binary decision controllers, finite state controllers · CPC title

• G05B13/0265

  the criterion being a learning criterion · CPC title

• G10L15/142

  Hidden Markov Models [HMMs] · CPC title