Apparatus and methods for context determination using real time sensor data

Patent metadata

Field	Value
Publication number	US-9821470-B2
Application number	US-201414489368-A
Country	US
Kind code	B2
Filing date	Sep 17, 2014
Priority date	Sep 17, 2014
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

Computerized appliances may be operated by users remotely. In one exemplary implementation, a learning controller apparatus may be operated to determine association between a user indication and an action by the appliance. The user indications, e.g., gestures, posture changes, audio signals may trigger an event associated with the controller. The event may be linked to a plurality of instructions configured to communicate a command to the appliance. The learning apparatus may receive sensory input conveying information about robot's state and environment (context). The sensory input may be used to determine the user indications. During operation, upon determine the indication using sensory input, the controller may cause execution of the respective instructions in order to trigger action by the appliance. Device animation methodology may enable users to operate computerized appliances using gestures, voice commands, posture changes, and/or other customized control elements.

First claim

What is claimed:

1. A computerized method for providing a remote control command to a computerized device based on a sequence of digital images, the method comprising:
determining a discrepancy measure based on a comparison of pixels of a current image of the sequence of digital images to a reference image;
determining a salient feature based on an analysis of the discrepancy measure, the salient feature being associated with a portion of the pixels within the current image; and
based on an existence of a previously established association between an occurrence of a user indication associated with an action by the computerized device and the salient feature, automatically transmitting a command to the computerized device, the command configured to cause the computerized device to execute the action;
wherein:
the salient feature comprises a representation of a user body portion;
the reference image comprises an image acquired prior to the current image without the representation of the user body portion; and
the reference image is based on a low pass filter operation on a plurality of images from the sequence of digital images, where the individual ones of the plurality of images precede the current image.

2. The computerized method of claim 1 , wherein:
the determining the discrepancy measure comprises determining a difference image based on a pixel wise difference operation between the current image and the reference image, and the difference image comprises one or more background pixel values and two or more areas of pixels having values different from the one or more background pixel values; and
the analysis of the discrepancy measure comprises selecting one area from the two or more areas with a winner takes all process.

3. The computerized method of claim 2 , wherein the winner takes all process is configured based on determining, for a given area of the two or more areas, one or more of: a total number of pixels within the given area, a sum of values of pixels within the given area, and a sum of deviations from a reference pixel value of a plurality of values of pixels within the given area.

4. The computerized method of claim 1 , further comprising receiving the sequence of digital images from a camera that is remote from the computerized device.

5. The computerized method of claim 4 , further comprising:
storing information related to one or more user indications, each of the one or more user indications corresponding to a state;
wherein
the command is associated with the one or more user indications; and
the automatically transmitting the command is based on an occurrence of the state.

6. The computerized method of claim 5 , wherein the state comprises a position of the computerized device in an environment of the computerized device.

7. The computerized method of claim 1 , further comprising:
characterizing an environment external to the computerized device; and
conveying one or more of a position, a gesture, or a movement of a person via a parameter.

8. The computerized method of claim 1 , further comprising:
acquiring the reference image prior to the current image without the representation of the user body portion; and
wherein the salient feature comprises a representation of a user body portion.

9. The computerized method of claim 1 , wherein the automatically transmitting the command comprises transmitting via an infrared signal.

10. The computerized method of claim 1 , wherein the automatically transmitting the command comprises transmitting via a radio frequency signal.

11. The computerized method of claim 1 , further comprising:
executing the action within an environment comprising one or more objects characterized by one or more corresponding object motion parameters;
the executed action is characterized by a motion parameter of a robotic device; and
the executed action is configured based on a modification of the motion parameter based on a value associated with a corresponding one of the one or more object motion parameters.

12. The computerized method of claim 1 , wherein:
the computerized device comprises a household appliance configured to perform a cleaning task of a user premises; and
a state comprises information related to the user premises.

13. The computerized method of claim 12 , further comprising:
loading another set of instructions which were previously configured to cause execution of another task by another computerized device;
wherein the loading the another set of instructions is triggered automatically by the computerized device based on a characteristic of the premises.

14. A computerized method for providing a remote control command to a computerized device based on a sequence of digital images, the method comprising:
determining a discrepancy measure based on a comparison of pixels of a current image of the sequence of digital images to a reference image;
determining a salient feature based on an analysis of the discrepancy measure, the salient feature being associated with a portion of the pixels within the current image;
based on an existence of a previously established association between an occurrence of a user indication associated with an action by the computerized device and the salient feature, automatically transmitting a command to the computerized device, the command configured to cause the computerized device to execute the action; and
loading another set of instructions which were previously configured to cause execution of another task by another computerized device;
wherein:
the computerized device comprises a household appliance configured to perform a cleaning task of a user premises;
a state comprises information related to the user premises; and
the loading the another set of instructions is triggered automatically by the computerized device based on a characteristic of the user premises.

15. The computerized method of claim 14 , wherein the reference image is based on a low pass filter operation on a plurality of images from the sequence of digital images, where the individual ones of the plurality of images precede the current image.

16. The computerized method of claim 15 , wherein:
the determining the discrepancy measure comprises determining a difference image based on a pixel wise difference operation between the current image and the reference image, and the difference image comprises one or more background pixel values and two or more areas of pixels having values different from the one or more background pixel values; and
the analysis of the discrepancy measure comprises selecting one area from the two or more areas with a winner takes all process configured to select one area from the two or more areas.

17. The computerized method of claim 16 , wherein the winner takes all process is configured based on determining, for a given area of the two or more areas, one or more of: a total number of pixels within the given area, a sum of values of pixels within the given area, and a sum of deviations from a reference pixel value of a plurality of values of pixels within the given area.

18. The computerized method of claim 14 , further comprising receiving the sequence of digital images from a camera that is remote from the computerized device.

19. The computerized method of claim 14 , further comprising:
characterizing an environment external to the computerized device; and
conveying one or more of a position, a gesture, or a movement of a person via a parameter.

20. The computerized method of clai

Assignees

• Brain Corp

Inventors

• Laurent Patryk
• Petre Csaba
• Izhikevich Eugene M

Classifications

• B25J13/006 Primary

  by means of a wireless system for controlling one or several manipulators · CPC title

• G05B2219/2642

  Domotique, domestic, home control, automation, smart house · CPC title

• G05B15/02

  electric · CPC title

• Y10S901/03

  Teaching system · CPC title