Wearable computer with superimposed controls and instructions for external device

Patent metadata

Field	Value
Publication number	US-8941560-B2
Application number	US-201113238557-A
Country	US
Kind code	B2
Filing date	Sep 21, 2011
Priority date	Sep 21, 2011
Publication date	Jan 27, 2015
Grant date	Jan 27, 2015

Abstract

A wearable computing device includes a head-mounted display (HMD) that provides a field of view in which at least a portion of the environment of the wearable computing device is viewable. The HMD is operable to display images superimposed over the field of view. When the wearable computing device determines that a target device is within its environment, the wearable computing device obtains target device information related to the target device. The target device information may include information that defines a virtual control interface for controlling the target device and an identification of a defined area of the target device on which the virtual control image is to be provided. The wearable computing device controls the HMD to display the virtual control image as an image superimposed over the defined area of the target device in the field of view.

First claim

What is claimed is:

1. A method comprising:
determining that a controllable target device is within an environment of a wearable computing device comprising a head-mounted display (HMD), wherein the HMD provides a field of view in which at least a portion of the environment is viewable, and wherein the HMD is operable to display images superimposed over the field of view;
obtaining target device information related to the controllable target device, wherein the target device information defines a virtual control interface for controlling the controllable target device and identifies a defined area of the controllable target device on which the virtual control interface is to be provided, wherein the target device information further includes status information related to an operational status of the controllable target device;
controlling the HMD to display the virtual control interface as an image superimposed over the defined area of the controllable target device in the field of view and to display an indication of the operational status of the controllable target device;
recognizing a gesture corresponding to interaction with the virtual control interface superimposed over the defined area of the controllable target device;
recognizing a control instruction for the controllable target device based on the recognized gesture; and
transmitting the recognized control instruction to the controllable target device.

2. The method of claim 1 , wherein the wearable computing device further comprises a camera configured to obtain images from a point-of-view.

3. The method of claim 2 , wherein the point-of-view of the camera substantially corresponds to the field of view provided by the HMD.

4. The method of claim 2 , wherein determining that a controllable target device is within an environment of a wearable computing device comprises:
receiving a point-of-view image obtained by the camera; and
identifying the controllable target device in the point-of-view image.

5. The method of claim 4 , wherein the point-of-view image is a video image.

6. The method of claim 1 , wherein determining that a controllable target device is within an environment of a wearable computing device comprises:
receiving a beacon from the controllable target device.

7. The method of claim 1 , wherein determining that a controllable target device is within an environment of a wearable computing device comprises:
recognizing a radio frequency identification (RFID) tag on the controllable target device.

8. The method of claim 1 , wherein determining that a controllable target device is within an environment of a wearable computing device comprises:
recognizing an optical identification on the controllable target device.

9. The method of claim 8 , wherein the optical identification is a barcode or a quick response (QR) code.

10. The method of claim 1 , wherein determining that a controllable target device is within an environment of a wearable computing device comprises:
the wearable computing device obtaining environment information regarding the environment of the wearable computing device; and
the wearable computing device identifying the controllable target device based on the environment information.

11. The method of claim 1 , wherein determining that a controllable target device is within an environment of a wearable computing device comprises:
the wearable computing device obtaining environment information regarding the environment of the wearable computing device;
the wearable computing device transmitting the environment information to a server network; and
the wearable computing device receiving an identification of the controllable target device from the server network.

12. The method of claim 11 , wherein the environment information comprises a physical location of the wearable computing device.

13. The method of claim 1 , wherein obtaining target device information related to the controllable target device comprises:
the wearable computing device retrieving the target device information from a target device database.

14. The method of claim 1 , wherein obtaining target device information related to the controllable target device comprises:
the wearable computing device receiving the target device information from a server network.

15. The method of claim 1 , wherein the defined area of the controllable target device is an external surface of the controllable target device.

16. The method of claim 15 , further comprising:
determining a perspective appearance of the defined area in the field of view; and
generating the image of the virtual control interface based on the perspective appearance of the defined area in the field of view.

17. The method of claim 15 , further comprising:
determining a distance from the HMD to the defined area of the controllable target device; and
generating the image of the virtual control interface based on the distance.

18. The method of claim 1 , wherein transmitting the recognized control instruction to the controllable target device comprises the wireless computing device transmitting the recognized control instruction to the controllable target device via a WiFi, Bluetooth, or infrared link.

19. The method of claim 1 , wherein transmitting the recognized control instruction to the controllable target device comprises the wireless computing device transmitting the recognized control instruction to a network for transmission to the controllable target device.

20. The method of claim 1 , further comprising:
the HMD displaying instructions related to the controllable target device.

21. The method of claim 1 , wherein the virtual control interface comprises a plurality of visual elements and each visual element is associated with a respective control instruction for the controllable target device.

22. The method of claim 21 , wherein the recognized gesture corresponds to interaction with a particular visual element of the virtual control interface, and wherein the particular visual element is associated with the recognized control instruction.

23. The method of claim 1 , further comprising:
receiving the status information from the controllable target device.

24. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:
determining that a controllable target device is within a field of view provided by a see-through display controlled by the computing device;
obtaining target device information related to the controllable target device, wherein the target device information defines a virtual control interface for controlling the controllable target device and identifies a defined area of the controllable target device on which the virtual control interface is to be provided, wherein the target device information further includes status information related to an operational status of the controllable target device;
controlling the see-through display to display the virtual control interface as an image superimposed over the defined area of the controllable target device in the field of view and to display an indication of the operational status of the controllable target device;
recognizing a gesture corresponding to interaction with the virtual control interface superimposed over the defined area of the controllable target device;
recognizing a control instruction for the con

Assignees

• Wong Adrian
• Miao Xiaoyu
• Google Inc

Inventors

• Wong Adrian
• Miao Xiaoyu

Classifications

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F1/163 Primary

  Wearable computers, e.g. on a belt · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06F2200/1613

  Supporting arrangements, e.g. for filters or documents associated to a laptop display · CPC title

• G06F3/14 Primary

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title