Sharing neighboring map data across devices

Patent metadata

Field	Value
Publication number	US-10466953-B2
Application number	US-201715593147-A
Country	US
Kind code	B2
Filing date	May 11, 2017
Priority date	Mar 30, 2017
Publication date	Nov 5, 2019
Grant date	Nov 5, 2019

Abstract

A computing device and method are provided for transmitting a relevant subset of map data, called a neighborhood, to enable mutual spatial understanding by multiple display devices around a target virtual location to display a shared hologram in the same exact location in the physical environment at the same moment in time. The computing device may comprise a processor, a memory operatively coupled to the processor, and an anchor transfer program stored in the memory and executed by the processor.

First claim

The invention claimed is:

1. A server computing device, comprising:
a processor;
a non-volatile storage device operatively coupled to the processor; and
an anchor transfer program stored in the non-volatile storage device and executed by the processor of the computing device, wherein
the anchor transfer program is configured to:
receive a transfer request from a first display device to transfer anchor data of a target virtual place-located anchor of one or more holograms at a target virtual location to a second display device, the transfer request initiated by a user via the first display device and specifying the target virtual place-located anchor, the anchor data comprising neighboring map data corresponding to the target virtual location for the specified target virtual place-located anchor, the neighboring map data being map data of a neighborhood in a vicinity around the target virtual location including pose graphs created by sensor measurements performed by the first display device;
retrieve and transmit the anchor data to the second display device responsive to the transfer request;
cause the second display device to incorporate, via a stitching process, the neighboring map data into existing map data of the second display device, the stitching process including stitching together the pose graphs of the neighboring map data created by the first display device and the existing map data of the second display device by applying a common global coordinate system corresponding to an aligned three-dimensional coordinate space of the neighborhood; and
cause the second display device to display the one or more holograms at the target virtual place-located anchor at the target virtual location from a vantage point of the second display device based on the incorporated map data and the existing map data of the second display device.

2. The server computing device of claim 1 , further comprising:
an anchor program stored in the non-volatile storage device and executed by the processor of the computing device, wherein
the anchor program is configured to:
receive an instruction from the first display device to generate the target virtual place-located anchor at the target virtual location;
generate the target virtual place-located anchor at the target virtual location; and
send the target virtual place-located anchor to the first display device.

3. The server computing device of claim 1 , wherein
the anchor transfer program is configured to transmit to the first display device neighboring map data in a serialized format; and
the second display device receives the neighboring map data in a deserialized format.

4. The server computing device of claim 1 , wherein
the target virtual location is world-locked to a position that is fixed in a three-dimensional coordinate space overlaid upon a real world three-dimensional environment.

5. The server computing device of claim 1 , wherein
the target virtual location is world-locked to a position relative to an object in a real world three-dimensional environment.

6. The server computing device of claim 1 , wherein
the neighboring map data comprises keyframes and at least a portion of pose-graphs describing rotational and translational motion of the display devices through a real world three-dimensional environment.

7. The server computing device of claim 6 , further comprising:
visual sensors and/or inertial measurement sensors, wherein
the visual sensors and/or inertial measurement sensors track the rotational and translational motion of the display devices for the keyframes and the at least a portion of pose-graphs.

8. The server computing device of claim 6 , wherein
the keyframes comprise at least one of a fingerprint of a Wi-Fi beacon, gravity data, temperature data, global positioning data, and calibration data.

9. A method, comprising:
receiving a transfer request from a first display device to transfer anchor data of a target virtual place-located anchor of one or more holograms at a target virtual location to a second display device, the transfer request initiated by a user via the first display device and specifying the target virtual place-located anchor, the anchor data comprising neighboring map data corresponding to the target virtual location for the specified target virtual place-located anchor, the neighboring map data being map data of a neighborhood in a vicinity around the target virtual location including pose graphs created by sensor measurements performed by the first display device;
retrieving and transmitting the anchor data to the second display device responsive to the transfer request; and
subsequent to the first display device transferring anchor data to the second display device, causing the second display device to incorporate, via a stitching process, the neighboring map data into existing map data of the second display device, and causing the second display device to display the one or more holograms at the target virtual place-located anchor at the target virtual location from a vantage point of the second display device based on the incorporated map data and the existing map data of the second display device, the stitching process including stitching together the pose graphs of the neighboring map data created by the first display device and the existing map data of the second display device by applying a common global coordinate system corresponding to an aligned three-dimensional coordinate space of the neighborhood.

10. The method of claim 9 , further comprising:
receiving an instruction from the first display device to generate the target virtual place-located anchor at the target virtual location;
generating the target virtual place-located anchor at the target virtual location; and
sending the target virtual place-located anchor to the first display device.

11. The method of claim 9 , wherein
the neighboring map data is transmitted to the first display device in a serialized format, and
the second display device receives the neighboring map data in a deserialized format.

12. The method of claim 9 , wherein
the target virtual location is world-locked to a position that is fixed in a three-dimensional coordinate space overlaid upon a real world three-dimensional environment.

13. The method of claim 9 , wherein
the target virtual location is world-locked to a position relative to an object in a real world three-dimensional environment.

14. The method of claim 9 , wherein
the neighboring map data comprises keyframes and pose-graphs recording rotational and translational motion of the display devices through a real world three-dimensional environment.

15. The method of claim 14 , wherein
visual sensors and/or inertial measurement sensors track the rotational and translational motion of the display devices for the keyframes and pose-graphs.

16. The method of claim 14 , wherein
the keyframes comprise at least one of a fingerprint of a Wi-Fi beacon, gravity data, temperature data, global positioning data, and calibration data.

17. A server computing device, comprising:
a processor;
a non-volatile storage device operatively coupled to the processor; and
an anchor transfer program stored in the non-volatile storage device and executed by the processor of the computing device, wherein
the anchor transfer program is configured to:
receive a transfer request from a first display device to transfer anchor data of a target virtual place-located anchor of one or more holograms at a target virtual location to a second display device, the transfer request in

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Eade Ethan
• Vanturennout Jeroen
• Lyons Jonathan
• Fields David
• Lazarow Gavin Dean
• Bhatnagar Tushar Cyril

Classifications

• G03H2226/05

  Means for tracking the observer · CPC title

• G03H2226/04

  Transmission or communication means, e.g. internet protocol · CPC title

• G03H2226/02

  Computing or processing means, e.g. digital signal processor [DSP] · CPC title

• G03H2001/2252

  Location of the holobject · CPC title

• G03H2001/0088

  for video-holography, i.e. integrating hologram acquisition, transmission and display · CPC title