Identification of augmented reality image display position

Patent metadata

Field	Value
Publication number	US-10338392-B2
Application number	US-201715409472-A
Country	US
Kind code	B2
Filing date	Jan 18, 2017
Priority date	Jun 13, 2016
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

A head-mounted display (HMD) device may include an at least partially see-through display, one or more location sensors, a communication unit, and a processor. The HMD device may determine at least a location of the one or more location sensors in a physical environment. The communication unit may receive peer location data from at least one peer HMD device in the physical environment. The processor of the HMD device may determine a field of view of the HMD device, identify a display position at which the augmented reality image can be displayed and viewed in the physical environment by the HMD device and the at least one peer HMD device, and display the augmented reality image at the display position on the at least partially see-through display of the HMD device.

First claim

The invention claimed is:

1. A head-mounted display (HMD) device comprising:
an at least partially see-through display configured to display an augmented reality image representing a virtual object, the virtual object having at least a first portion, a second portion, and an associated script for animating the virtual object;
one or more location sensors from which the HMD device determines at least a location of the HMD device in a physical environment;
a communication unit configured to receive peer location data from at least one peer HMD device in the physical environment, wherein the peer location data indicates a peer field of view of the at least one peer HMD device, the peer field of view comprising a range within which augmented reality images may be presented; and
a processor configured to:
based at least on the received peer location data, identify a display position at which the virtual object is to be displayed and viewed in the physical environment by the HMD device and the at least one peer HMD device;
determine a shared focal point based on the display position, the location of the HMD device, and the peer location data;
determine a first orientation of the first portion of the virtual object at the display position based on the shared focal point and focal point information of the script, the first orientation being common to the HMD device and the at least one peer HMD device;
determine a second orientation of the second portion of the virtual object at the display position based on the focal point information and a device-specific focal point that corresponds to the location of the HMD device relative to the display position, the second orientation being specific to the location of the HMD device and differing from the at least one peer HMD device; and
display and animate the augmented reality image of the virtual object at the display position with the first portion at the first orientation and the second portion at the second orientation on the at least partially see-through display of the HMD device according to the script.

2. The HMD device of claim 1 , wherein peer location data includes at least one of surface data about surfaces and candidate positions at which the augmented reality image can be displayed and viewed in the physical environment by the HMD device.

3. The HMD device of claim 1 , wherein the communication unit is further configured to transmit an enumeration request to determine if one or more proximal HMD devices are in the physical environment; and
receive an enumeration response from the one or more proximal HMD devices indicating that the one or more proximal HMD devices are located in the physical environment.

4. The HMD device of claim 3 , wherein the processor is further configured to identify the at least one peer HMD device from one or more proximal HMD devices based on one or more logical constraints.

5. The HMD device of claim 4 , wherein the one or more logical constraints include at least one of: the one or more proximal HMD devices being within a predetermined distance of the determined location, the one or more proximal HMD devices being within a predetermined area, and the one or more proximal HMD devices belonging to a predetermined group.

6. The HMD device of claim 1 , wherein the communication unit is configured to transmit location data about the HMD device to the at least one peer HMD device.

7. The HMD device of claim 1 , wherein the processor is further configured to:
identify candidate positions at which the augmented reality image can be displayed and viewed in the physical environment by the HMD device.

8. The HMD device of claim 7 , wherein the received peer location data comprises peer candidate positions at which the augmented reality image can be displayed and viewed in the physical environment by the at least one peer HMD device.

9. The HMD device of claim 8 , wherein the processor is further configured to:
identify compatible positions at which the augmented reality image can be displayed and viewed in the physical environment by the HMD device and the at least one peer HMD device by comparing peer candidate positions to the candidate positions and determining overlapping positions visible by both the HMD device and the at least one peer HMD device; and
select the display position from the compatible positions based on one or more logical constraints.

10. The HMD device of claim 1 , further comprising an optical sensor configured to acquire surface data about surfaces.

11. The HMD device of claim 10 , wherein the received peer location data comprises peer surface data about surfaces within the peer field of view of the at least one peer HMD device.

12. The HMD device of claim 11 , wherein the processor is further configured to:
identify compatible positions at which the augmented reality image can be displayed and viewed in the physical environment by the HMD device and the at least one peer HMD device by creating a union of the surface data and the received peer surface data and determining overlapping positions visible by both the HMD device and the at least one peer HMD device; and
select the display position from the compatible positions based on one or more logical constraints.

13. The HMD device of claim 1 , wherein the augmented reality image is a virtual presenter, and the processor is further configured to:
determine a plurality of focal points based at least on one of: the display position, the location of the HMD device, and the peer location data; and
animate the virtual presenter to perform the script, wherein the script comprises gestures and focal point information about the plurality of focal points, and the gestures of the virtual presenter depend at least on the focal point information.

14. The HMD device of claim 1 , wherein the augmented reality image is a virtual presenter, and the processor is further configured to:
display a presentation image at a presentation position, the presentation position being based on the location of the HMD device, the display position, and an area occupied by the virtual presenter; and
determine a plurality of focal points based at least on one of: the display position, the presentation position, the location of the HMD device, and the peer location data; and
animate the virtual presenter to perform the script, wherein the script comprises gestures and focal point information about the plurality of focal points, and the gestures of the virtual presenter depend at least on the focal point information.

15. The HMD device of claim 1 , wherein the communication unit is further configured to determine that an additional HMD device has entered the physical environment, receive additional location data from the additional HMD device, and wherein the processor is further configured to transmit an instruction to the additional HMD device indicating an unoccupied position nearest to the additional HMD device at which the augmented reality image displayed at the display position can be viewed by the additional HMD device.

16. A method implemented at least in part by a head-mounted display (HMD) device, the method comprising:
determining a location of the HMD device in a physical environment;
receiving peer location data from at least one peer HMD device in the physical environment via a communication unit of the HMD device, wherein the peer location data indicates a peer field of view of the at least one peer HMD device, the peer field of view comprising a range within which augmented reality images may be presented;
with a processor of the HMD device:
based at le

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Kohler Jeffrey
• Wright Shawn Crispin
• Mcmanus Michael W
• Ambrus Anthony
• Krauskopf Korey

Classifications

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title