Localizing devices in augmented reality environment

Patent metadata

Field	Value
Publication number	US-9865091-B2
Application number	US-201514843596-A
Country	US
Kind code	B2
Filing date	Sep 2, 2015
Priority date	Sep 2, 2015
Publication date	Jan 9, 2018
Grant date	Jan 9, 2018

Abstract

Examples are disclosed herein that relate to identifying and localizing devices in an environment via an augmented reality display device. One example provides, on a portable augmented reality computing device, a method including establishing a coordinate frame for an environment, and discovering, via a location-sensitive input device, a location of a physical manifestation of the device in the environment, assigning a device location for the device in the coordinate frame based upon the location of the physical manifestation, and modifying an output of the portable augmented reality computing device based upon a change in relative position between the portable augmented reality computing device and the physical manifestation in environment.

First claim

The invention claimed is:

1. On a portable augmented reality computing device, a method comprising:
establishing a coordinate frame for an environment;
discovering, via a location-sensitive input device, a location of a physical manifestation of a device in the environment;
assigning a device location for the device in the coordinate frame based upon the location of the physical manifestation;
modifying an augmented reality image displayed by the portable augmented reality computing device based upon a change in relative position between the portable augmented reality computing device and the device in the environment by modifying a portion of the augmented reality image that corresponds to the device location to reduce occlusion of a view of the device in the environment by the augmented reality image.

2. The method of claim 1 , wherein the location-sensitive input device comprises one or more of an image sensor and a microphone array.

3. The method of claim 1 , wherein discovering the location of the physical manifestation comprises detecting a light controlled by the device in the environment.

4. The method of claim 1 , wherein discovering the location of the physical manifestation comprises detecting motion of an object controlled by the device.

5. The method of claim 1 , wherein discovering the location of the physical manifestation comprises detecting a location of a display screen.

6. The method of claim 1 , further comprising discovering the device on a wireless network.

7. The method of claim 1 , wherein the output is an augmented reality image, and wherein modifying the output of the portable augmented reality computing device comprises modifying the augmented reality image to avoid occluding the physical manifestation with the augmented reality image.

8. The method of claim 7 , further comprising tracking motion of the portable augmented reality computing device and updating the augmented reality image based upon the tracking to avoid occluding the physical manifestation.

9. An augmented reality display device, comprising:
a see-through display device;
a location-sensitive input device;
a logic device; and
a storage device comprising instructions executable by the logic device to
establish a coordinate frame for an environment based upon data from the location-sensitive input device,
conduct communication with a device to direct the device to alter a physical manifestation of the device in the environment, the physical manifestation being detectable by the location-sensitive input device,
detect, via the location-sensitive input device, a location of the physical manifestation based on an alteration of the physical manifestation of the device in the environment,
assign a device location for the device in the coordinate frame based upon the location of the physical manifestation, and
modify an output of the augmented reality display device based upon a change in relative position between the augmented reality display device and the physical manifestation in the environment by modifying a portion of an augmented reality image that corresponds to the device location to reduce occlusion of a view of the device in the environment by the augmented reality image.

10. The augmented reality display device of claim 9 , wherein the location-sensitive input device comprises one or more of an image sensor and a microphone array.

11. The augmented reality display device of claim 9 , wherein the physical manifestation comprises a light controlled by the device in the environment.

12. The augmented reality display device of claim 9 , wherein the physical manifestation comprises a motion of an object controlled by the device in the environment.

13. The augmented reality display device of claim 9 , wherein the physical manifestation comprises a display screen.

14. The augmented reality display device of claim 9 , wherein the instructions are further executable to discover the device on a wireless network.

15. The augmented reality display device of claim 9 , wherein the output is an augmented reality image, and wherein the instructions are executable to modify the augmented reality image to avoid occluding the physical manifestation with the augmented reality image.

16. The augmented reality display device of claim 9 , further comprising a motion sensor configured to track motion of the augmented reality display device.

17. The augmented reality display device of claim 9 , wherein the augmented reality display device comprises a head-mounted display device.

18. A computing device, comprising:
a logic device; and
a storage device comprising instructions executable by the logic device to
establish a connection with a portable augmented reality display device remote from the computing device,
receive instruction from the portable augmented reality display device requesting the computing device to control each of a plurality of devices in the environment to alter a physical manifestation of the device in the environment,
send to the portable augmented reality display device information regarding a device address of each device in the environment,
send instruction to a first device of the plurality of devices in the environment to alter a physical manifestation output by the first device that is detectable by a location-sensitive input device of the portable augmented reality display device,
receive confirmation from the portable augmented reality display device confirming discovery of the first device,
send instruction to a second device of the plurality of devices in the environment to alter a physical manifestation output by the second device that is detectable by the location-sensitive input device of the portable augmented reality display device, and
receive confirmation from the portable augmented reality display device confirming discovery of the second device.

19. The computing device of claim 18 , wherein the instructions are further executable to control the plurality of devices in the environment to alter the physical manifestations in a specified order.

20. The computing device of claim 18 , wherein the instructions are executable to receive a request from the portable augmented reality display device for the device address of each device in the environment in a specified order.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Giraldi Riccardo
• Gavriliuc Anatolie
• Chua Michelle
• Muehlhausen Andrew Frederick
• Held Robert Thomas
• Heuvel Joseph Van Den
• Omotani Todd Alan
• Wifall Richard J
• Sadak Christian
• Alt Gregory

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• H04R29/005

  Microphone arrays · CPC title