Providing multiplayer augmented reality experiences

Patent metadata

Field	Value
Publication number	US-10773169-B2
Application number	US-201815876819-A
Country	US
Kind code	B2
Filing date	Jan 22, 2018
Priority date	Jan 22, 2018
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

Systems and methods are described for providing co-presence in an augmented reality environment. The method may include controlling a first and second computing device to detect at least one plane associated with a scene of the augmented reality environment generated for a physical space, receiving, from the first computing device, a first selection of a first location within the scene and a first selection of a second location within the scene, generating a first reference marker corresponding to the first location and generating a second reference marker corresponding to the second location, receiving, from a second computing device, a second selection of the first location within the scene and a second selection of the second location within the scene, generating a reference frame and providing the reference frame to the first computing device and to the second computing device to establish co-presence in the augmented reality environment.

First claim

What is claimed is:

1. A computer-implemented method for providing co-presence in an augmented reality environment, the method comprising:
controlling a first computing device and a second computing device to detect at least one plane associated with a scene of the augmented reality environment generated for a physical space;
receiving, from the first computing device, a first selection of a first location within the scene and a first selection of a second location within the scene;
generating a first reference marker corresponding to the first location and generating a second reference marker corresponding to the second location;
receiving, from a second computing device, a second selection of the first location within the scene and a second selection of the second location within the scene;
generating a reference frame centered at the first reference marker that indicates a direction toward the second reference marker, the reference frame being generated using the at least one plane, the first location, and the second location; and
providing the reference frame to the first computing device and to the second computing device to establish co-presence in the augmented reality environment, establishing the co-presence including generating, for the scene, a registration of the first computing device relative to the second computing device.

2. The computer-implemented method of claim 1 , wherein the reference frame is generated based on a detected pose associated with the first computing device that selected the first location and the second location and a determined intersection between the second location and the detected plane.

3. The computer-implemented method of claim 1 , further comprising:
in response to receiving a third selection of the first location and a third selection of the second location from a third computing device, providing the reference frame to the third computing device to establish co-presence in the augmented reality environment, establishing the co-presence including generating, for the scene:
a registration of the third computing device relative to the first computing device; and
a registration of the third computing device relative to the second computing device.

4. The computer-implemented method of claim 1 , wherein receiving, at a second computing device, a selection of the first location within the scene and a selection of the second location within the scene includes automatically detecting, at the second computing device, the first reference marker and the second reference marker.

5. The computer-implemented method of claim 1 , further comprising:
displaying in the first computing device the first reference marker corresponding to the first location and displaying in the first computing device, the second reference marker corresponding to the second location, and
wherein receiving, at the first computing device, a selection of a first location within the scene and a selection of a second location within the scene is triggered by prompts received at a display device associated with the first computing device.

6. The computer-implemented method of claim 1 , wherein the established co-presence is used to access an application in the augmented reality environment and an application state is stored with the reference frame; and
re-establishing the reference frame includes the first computing device, the second computing device, or another computing device selecting upon the first location and the second location to gain access to the application according to the stored application state.

7. The method of claim 1 , wherein the first reference marker and the second reference marker define at least a portion of the reference frame with five degrees of freedom and wherein an angle generated by the indicated direction and the at least one plane define a sixth degree of freedom for the reference frame.

8. The computer-implemented method of claim 1 , wherein the first location represents a first physical feature in the physical environment and the second location represents a second physical feature in the physical environment, the first physical feature and the second physical feature being agreed upon between a user associated with the first computing device and a user associated with the second computing device.

9. The computer-implemented method of claim 1 , wherein the co-presence is established without using position data associated with the first computing device or the second computing device.

10. A computer program product tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to:
control a first computing device and a second computing device to detect at least one plane associated with a scene of an augmented reality environment generated for a physical space;
receive, at the first computing device, a selection of a first location within the scene and a selection of a second location within the scene;
generate a first reference marker corresponding to the first location and generating a second reference marker corresponding to the second location;
generate a reference frame centered at the first reference marker pointed in a direction of the second reference marker, the reference frame being generated based at least in part on the at least one plane, the first location, and the second location; and
provide the reference frame to the first computing device and to the second computing device to establish co-presence in the augmented reality environment.

11. The computer program product of claim 10 , wherein the computing device is further caused to receive, at a second computing device, a selection of the first location within the scene and a selection of the second location within the scene.

12. The computer program product of claim 10 , wherein the reference frame is used to re-establish the co-presence between the first computing device and the second computing device, in response to losing a connection to augmented reality environment.

13. The computer program product of claim 10 , wherein the reference frame is used to re-establish the co-presence between the first computing device and the second computing device, in response to changing the location of the physical space associated with the augmented reality environment.

14. A system comprising:
at least one processor;
memory storing instructions that, when executed by the at least one processor, cause the system to perform operations including,
controlling a first computing device and a second computing device to detect at least one plane associated with a scene of an augmented reality environment generated for a physical space;
receiving, at the first computing device, a selection of a first location within the scene and a selection of a second location within the scene;
generating a first reference marker corresponding to the first location and generating a second reference marker corresponding to the second location;
receiving, at a second computing device, a selection of the first location within the scene and a selection of the second location within the scene;
generating a reference frame centered at the first reference marker pointed in a direction of the second reference marker, the reference frame being generated using the at least one plane, the first location, and the second location; and
providing the reference frame to the first computing device and to the second computing device to establish co-presence in the augmented reality environment, the co-presence includi

Assignees

• Google Llc

Inventors

• Leeper Adam
• Ullman John
• Yang Cheng
• Tan Peter

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A63F13/65 Primary

  automatically by game devices or servers from real world data, e.g. measurement in live racing competition · CPC title

• G06V10/24

  Aligning, centring, orientation detection or correction of the image · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title

• H04L67/131

  Protocols for games, networked simulations or virtual reality · CPC title