Facilitating device control

Patent metadata

Field	Value
Publication number	US-10216177-B2
Application number	US-201615051180-A
Country	US
Kind code	B2
Filing date	Feb 23, 2016
Priority date	Feb 23, 2015
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

A method of deriving autonomous control information involves receiving one or more sets of associated environment sensor information and device control instructions. Each set of associated environment sensor information and device control instructions includes environment sensor information representing an environment associated with an operator controllable device and associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information. The method also involves deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, the autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, the autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

First claim

The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. A method of deriving autonomous control information, the method comprising:
receiving one or more sets of associated environment sensor information and device control instructions, wherein each set of associated environment sensor information and device control instructions comprises:
environment sensor information representing an environment associated with an operator controllable device; and
associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information;
deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, said autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, said autonomous device control signals configured to cause the autonomous device to take at least one autonomous action, said autonomous control information comprising a representation of an artificial intelligence model derived from the one or more sets of associated environment sensor information and device control instructions.

2. The method of claim 1 further comprising:
receiving first autonomous environment sensor information representing an environment associated with a first autonomous device;
deriving first autonomous device control instructions from the first autonomous environment sensor information using the autonomous control information;
producing first autonomous device control signals representing the first autonomous device control instructions for causing the first autonomous device to take at least one autonomous action.

3. The method of claim 1 wherein receiving the one or more sets of associated environment sensor information and device control instructions comprises:
receiving first environment sensor information representing a first environment of a first operator controllable device;
producing first operator interface control signals derived from the first environment sensor information for causing a first operator interface to simulate the first environment for a first operator interacting with the first operator interface;
receiving first operator interface sensor information representing at least one action taken by the first operator in response to the first operator interface simulating the first environment;
deriving first device control instructions from the first operator interface sensor information, said first device control instructions configured to cause the first operator controllable device to simulate the at least one action taken by the first operator;
wherein one of the one or more sets of associated environment sensor information and device control instructions comprises the first environment information and the first device control instructions.

4. The method of claim 1 wherein the environment sensor information for each of the one or more sets of associated environment sensor information and device control instructions comprises at least one of the following:
visual information;
audio information;
location information;
feature proximity information;
force feedback information;
chemical information;
temperature information;
kinematic information; and
orientation information.

5. The method of claim 1 wherein the device control instructions for each of the one or more sets of associated environment sensor information and device control instructions represents signals for causing the operator controllable device to take at least one of the following actions:
gripping a feature;
pushing a feature;
pulling a feature;
propelling the operator controllable device;
moving the operator controllable device;
moving a feature; and
emitting communication signals.

6. The method of claim 1 , further comprising:
deriving, by the processor, the representation of the artificial intelligence model from a model comprising at least one of:
a conditional deep belief network;
a conditional restricted Boltzmann machine;
a long short-term memory neural network;
a recurrent neural network;
a deep hierarchical learning system; and
a variational auto-encoder.

7. A non-transitory computer-readable storage medium having stored thereon codes which, when executed by at least one processor, cause the at least one processor to:
receive one or more sets of associated environment sensor information and device control instructions, wherein each set of associated environment sensor information and device control instructions comprises:
environment sensor information representing an environment associated with an operator controllable device; and
associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information;
derive autonomous control information from the one or more sets of associated environment sensor information and device control instructions, said autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, said autonomous device control signals configured to cause the autonomous device to take at least one autonomous action, said autonomous control information comprising a representation of an artificial intelligence model derived from the one or more sets of associated environment sensor information and device control instructions.

8. The non-transitory computer-readable storage medium of claim 7 , wherein the stored codes cause the at least one processor to:
derive the autonomous control instructions from a representation of an artificial intelligence model comprising at least one of:
a conditional deep belief network;
a conditional restricted Boltzmann machine;
a long short-term memory neural network;
a recurrent neural network;
a deep hierarchical learning system; and
a variational auto-encoder.

9. The non-transitory computer-readable storage medium of claim 7 , wherein the environment sensor information for each of the one or more sets of associated environment sensor information and device control instructions comprises at least visual information.

10. A system for deriving autonomous control information, the system comprising:
means for receiving one or more sets of associated environment sensor information and device control instructions, wherein each set of associated environment sensor information and device control instructions comprises:
environment sensor information representing an environment associated with an operator controllable device; and
associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information;
means for deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, said autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, said auton

Assignees

• Kindred Sys Inc

Inventors

• Gildert Suzanne
• Rose George Samuel
• Taylor Graham William
• Bergstra James

Classifications

• G05B19/42 Primary

  Recording and playback systems, i.e. in which the program is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine · CPC title

• Y10S901/01

  Mobile robot · CPC title

• G05B2219/36184

  Record actions of human expert, teach by showing · CPC title

• G05B2219/36442

  Automatically teaching, teach by showing · CPC title