Apparatus and method for automatic discovery and suggesting personalized gesture control based on user's habit and context

Patent metadata

Field	Value
Publication number	US-9521239-B2
Application number	US-201414260922-A
Country	US
Kind code	B2
Filing date	Apr 24, 2014
Priority date	Apr 24, 2014
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

An apparatus and method for automatic discovery and suggesting personalized gestures from mobile terminal to control a plurality of devices in smart home system, based on user's habit and context. The method includes recording, via a mobile terminal, a context of the mobile terminal, recoding, via home system, actions taken by the plurality of devices in the home system, recognizing as a gesture command from a correlation of at least a portion of the recorded context of the user's mobile terminal with the recorded actions taken by the plurality of devices in the home system, and controlling at least one of the plurality of devices based a repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system.

First claim

What is claimed is:

1. A method for integrated discovery and control of a plurality of devices based on movement, context and/or gesture, the method comprising:
recording, via a mobile terminal, a context of a user of the mobile terminal;
recording, via home system, actions taken by the plurality of devices in the home system;
recognizing as a gesture command from a correlation of at least a portion of the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system; and
controlling at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system,
wherein the recorded context of the user of the mobile terminal includes macro movements recorded as a map movement and micro movements recorded as other movements of the user of the mobile terminal in relation to the user's external environment not recorded as the map movement.

2. The method of claim 1 , further comprising:
suggesting the gesture command based on at least a portion of the recognition of the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system; and
confirming, in response to the suggesting of the gesture command, the gesture command, the gesture command enables the controlling of the at least one of the plurality of devices in the home system.

3. The method of claim 1 , wherein the recording of the context of the user of the mobile terminal comprises recording at least a time, usage history and a physical movement of the mobile terminal.

4. The method of claim 3 , wherein the physical movement of the mobile terminal is determined via at least one of a Global Position System (GPS), WIFI hotspot, cell tower triangulation and a motion sensor.

5. The method of claim 2 , wherein the correlation of the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system comprises:
identifying the recorded actions taken by the at least one of the plurality of devices in the home system;
recognizing via a pattern classifier, a repeated pattern from the recorded context of the mobile terminal in correlation with a repeated pattern from the recorded actions taken by the at least one of the plurality of devices in the home system; and
matching, based on the pattern classifier, an action taken by the at least one of the plurality of devices in the home system with the recognized repeated pattern from the history recorded by the mobile terminal.

6. The method of claim 1 , wherein the controlling of the at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system comprises:
causing, when it is determined that a current context of a user of the mobile terminal are the same as the recorded context, and that the gesture command associated with the recorded patterns of the user of the mobile terminal, execution of an action associated with the gesture command of the user of the mobile terminal.

7. The method of claim 6 , wherein the repeated context of the user of the mobile terminal is walking up a walk way to a home of the user, stepping up several steps and pausing at a top of the several steps, and
wherein the controlling of at least one of the plurality of devices based on the user's repeated context comprises automatically opening a door of the user's home by a home system in communication with the mobile device.

8. An apparatus for integrated discovery and control of a plurality of devices based on movement and/or gesture, the apparatus comprising:
a mobile device recording unit configured to record a context of a user of the mobile terminal;
a home system recoding unit configured to record actions taken by the plurality of devices in the home system; and
a controller configured to:
recognize as a gesture command from a correlation of at least a portion of the recorded context of the user of the mobile terminal with the recorded actions taken by the at least one of the plurality of devices in the home system, and
control at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system,

wherein the recorded context of the user of the mobile terminal includes macro movements recorded as a map movement and micro movements recorded as other movements of the user of the mobile terminal in relation to the user's external environment not recorded as the map movement.

9. The apparatus of claim 8 , further comprising:
a gesture unit configured to suggest the gesture command based on at least a portion of the recognition of the recorded context of the user of the mobile terminal with the recorded actions taken by the at least one of plurality of devices in the home system; and
a confirmation unit configured to confirm, in response to the suggesting of the gesture command, the gesture command, the gesture command enables the controlling of the at least one the plurality of devices in the home system.

10. The apparatus of claim 8 , wherein mobile device recording unit records at least a time, usage history and a physical movement of the mobile terminal.

11. The apparatus of claim 10 , wherein the physical movement of the mobile terminal is determined via at least one of a Global Position System (GPS), WIFI hotspot, cell tower triangulation and a motion sensor.

12. The apparatus of claim 9 , wherein the controller correlating the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system comprises identifying the recorded actions taken by the at least one of the plurality of devices in the home system, recognizing via a pattern classifier, a repeated pattern from the recorded context of the mobile terminal in correlation with a repeated pattern from the recorded actions taken by the at least one of the plurality of devices in the home system, and matching, based on the pattern classifier, an action taken by the at least one of the plurality of devices in the home system with the recognized repeated pattern from the history recorded by the mobile terminal.

13. The apparatus of claim 8 , wherein the controller controlling the at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system comprises causing, when it is determined that a current context of a user of the mobile terminal are the same as the recorded context, and that the gesture command associated with the recorded patterns of the user of the mobile terminal, execution of an action associated with the gesture command of the user of the mobile terminal.

14. The apparatus of claim 13 , wherein, the controller upon detecting the repeated context of the user of the mobile terminal as walking up a walk way to a home of the user, stepping up several steps and pausing at a top of the several steps, controls at least one of the plurality of devices based on the user's confirming of the action is an automatic opening of a door of the user's home by a home system in communication with the mobile device.

15. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one pr

Assignees

• Samsung Electronics Co Ltd

Inventors

• Li Zhiyun

Classifications

• H04L12/2829

  involving user profiles according to which the execution of a home appliance functionality is automatically triggered · CPC title

• H04W4/025

  using location based information parameters · CPC title

• H04M11/007

  with remote control systems (remote control in general G08C) · CPC title

• H04M1/72533 Primary

  Electricity · mapped topic

• H04M1/72415 Primary

  for remote control of appliances · CPC title