Methods and Systems for Controlling External Devices Using a Wearable Apparatus

Patent metadata

Field	Value
Publication number	US-2017192401-A1
Application number	US-201715399571-A
Country	US
Kind code	A1
Filing date	Jan 5, 2017
Priority date	Jan 6, 2016
Publication date	Jul 6, 2017
Grant date	—

Abstract

A wearable apparatus is provided for capturing and processing images from an environment of a user. In one implementation, a system for controlling one or more controllable devices includes a transceiver and at least one processing device. The processing device is programmed to obtain one or more images captured by an image sensor included in a wearable apparatus, analyze the one or more images to identify a controllable device in an environment of a user of the wearable apparatus, analyze the one or more images to detect a visual trigger associated with the controllable device and, based on the detection of the visual trigger, transmit, via the transceiver, a command. The command may be configured to change at least one aspect of the controllable device.

First claim

What is claimed is:

1 . A system for controlling one or more controllable devices, the system comprising:
a transceiver; and at least one processing device programmed to:
obtain one or more images captured by an image sensor included in a wearable apparatus;
analyze the one or more images to identify a controllable device in an environment of a user of the wearable apparatus;
analyze the one or more images to detect a visual trigger associated with the controllable device; and
based on the detection of the visual trigger, transmit, via the transceiver, a command, the command being configured to change at least one aspect of the controllable device.

2 . The system of claim 1 , wherein the processing device is further programmed to identify a current state of the controllable device based on analysis of the one or more images.

3 . The system of claim 2 , wherein the processing device is further programmed to select the command from among a plurality of commands based on the identified current state of the controllable device.

4 . The system of claim 1 , wherein the processing device is further programmed to identify a context associated with the controllable device based on analysis of the one or more images.

5 . The system of claim 4 , wherein the processing device is further programmed to select values for one or more parameters associated with the command based on the identified context associated with the controllable device.

6 . The system of claim 5 , wherein the identified context is associated with a meal to be prepared held by the user, the controllable device includes a microwave, and the one or more parameters include settings for the microwave based on content of the meal.

7 . The system of claim 4 , wherein the context is associated with at least one of the visual trigger, an action involving the controllable device, an object held by the user, or a state of the controllable device.

8 . The system of claim 1 , wherein the processing device is further programmed to:
identify a type associated with the visual trigger; and select the command from among a plurality of commands based on the identified type of the visual trigger.

9 . The system of claim 8 , wherein the type of the visual trigger includes at least one of a hand gesture, an action involving the user's hand, or an action involving the controllable device.

10 . The system of claim 1 , wherein the visual trigger includes a hand gesture.

11 . The system of claim 10 , wherein the hand gesture includes an upward flick of an index finger, and the upward flick of the index finger is associated with a command for turning on a light switch.

12 . The system of claim 1 , wherein the system for controlling the one or more controllable devices is included in the wearable apparatus.

13 . The system of claim 1 , wherein the command is transmitted to a paired device.

14 . The system of claim 1 , wherein the at least one processing device is further programmed to:
obtain audio data captured by at least one audio sensor included in the wearable apparatus; and select the command from among a plurality of available commands based, at least in part, on the audio data.

15 . The system of claim 1 , wherein the at least one processing device is further programmed to:
receive, via the transceiver, an indication that the command has been received; and provide feedback to the user of the wearable apparatus indicating that the command has been received.

16 . The system of claim 1 , wherein the at least one processing device is further programmed to:
receive via the transceiver an indication that the at least one aspect of the controllable device has changed; and provide feedback to the user of the wearable apparatus indicating that the at least one aspect of the controllable device has changed.

17 . The system of claim 1 , wherein the at least one processing device is further programmed to:
obtain a second group of one or more images captured by the image sensor; analyze the second group of one or more images to identify a change in the at least one aspect of the controllable device; and provide feedback to the user of the wearable apparatus indicative of the change in the at least one aspect of the controllable device.

18 . The system of claim 1 , wherein transmitting the command comprises transmitting the command to the controllable device.

19 . The system of claim 1 , wherein the controllable device is associated with a locking mechanism, and the command is configured to cause the controllable device to lock or unlock the locking mechanism.

20 . The system of claim 1 , wherein the controllable device is associated with a light source, and the command is configured to cause the controllable device to change an illumination state of the light source.

21 . The system of claim 1 , wherein the controllable device is associated with at least one of a heating device, a ventilation device, an air conditioning device, or an HVAC system; and wherein the command is configured to change one or more settings of the heating device, the ventilation device, the air conditioning device, or the HVAC system.

22 . The system of claim 1 , wherein the controllable device is associated with at least one of: a computer, a printer, a television, a music player, a household appliance, a vehicle, a security system, or a toy.

23 . The system of claim 1 , further comprising a memory and, wherein, after analyzing the one or more images to identify the controllable device in the environment of the user of the wearable apparatus, the at least one processing device is further programmed to:
cause the transceiver to transmit an interrogation signal, the interrogation signal being configured to cause a change in at least one aspect of the controllable device; analyze one or more images captured by the image sensor after transmission of the interrogation signal to detect the change in the at least one aspect of the controllable device; and after detection of the change in the at least one aspect of the controllable device, store in the memory information relating to the controllable device.

24 . The system of claim 23 , wherein the information relating to the controllable device includes a location associated with the controllable device, a type associated with the controllable device, or at least one controllable function associated with the controllable device.

25 . The system of claim 23 , wherein the change in the at least one aspect of the controllable device includes a change in illumination of at least one light source associated with the controllable device.

26 . The system of claim 25 , wherein the at least one light source includes an infrared light source or a visible light source, and the change in illumination includes causing the at least one light source to at least one of: turn off, turn on, blink, dim, or brighten.

27 . The system of claim 1 , wherein the at least one processing device is further programmed to:
analyze one or more images captured by the image sensor to detect at least one recognized gesture made by the user; and cause a control signal to be transmitted to the controllable device, the control signal being configured to control at least one aspect of the controllable device, the at least one aspect of the controllable device to be controlled being associat

Assignees

• Orcam Technologies Ltd

Inventors

• Wexler Yonatan
• Shashua Amnon

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F18/24

  Classification techniques · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title