Providing multiplayer augmented reality experiences

Patent metadata

Field	Value
Publication number	US-11426663-B2
Application number	US-202016948009-A
Country	US
Kind code	B2
Filing date	Aug 27, 2020
Priority date	Jan 22, 2018
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

Systems and methods are described for providing co-presence in an augmented reality environment. The method may include controlling a first and second computing device to detect at least one plane associated with a scene of the augmented reality environment generated for a physical space, receiving, from the first computing device, a first selection of a first location within the scene and a first selection of a second location within the scene, generating a first reference marker corresponding to the first location and generating a second reference marker corresponding to the second location, receiving, from a second computing device, a second selection of the first location within the scene and a second selection of the second location within the scene, generating a reference frame and providing the reference frame to the first computing device and to the second computing device to establish co-presence in the augmented reality environment.

First claim

What is claimed is:

1. A computer-implemented method comprising:
controlling a first device and a second device to detect at least one plane associated with at least one scene in a session of a game provided in an augmented reality environment, the session provided as a service;
receiving, from the first device, a first selection of a first location within the at least one scene and a first selection of a second location within the at least one scene;
generating a first reference marker corresponding to the first location and generating a second reference marker corresponding to the second location;
receiving, from the second device, a second selection of the first location within the at least one scene and a second selection of the second location within the at least one scene;
generating a reference frame centered at the first reference marker that indicates a direction toward the second reference marker, the reference frame being generated using the at least one plane, the first location, and the second location; and
providing the reference frame to the first device and to the second device to establish co-presence in the session, wherein establishing the co-presence includes generating, for the session, a registration of the first device relative to the second device.

2. The computer-implemented method of claim 1 , wherein establishing co-presence in the session further includes the second device calculating a transformation between a tracked location of the first device and a pose associated with the reference frame.

3. The computer-implemented method of claim 1 , further comprising re-establishing the session for a selected physical space in response to detecting that at least the first device is located in a different physical space than the second device and the first device is requesting access to the session.

4. The computer-implemented method of claim 3 , wherein re-establishing the session for the selected physical space includes triggering initialization of an updated reference frame for the first device, the updated reference frame generated based at least in part on:
receiving, from the first device, selection of an updated first location and selection of an updated second location;
generating an updated first reference marker corresponding to the updated first location and generating an updated second reference marker corresponding to the updated second location;
correlating the first reference marker with the updated first reference marker;
correlating the second reference marker with the updated second reference marker; and
establishing the co-presence including generating, for the re-established session, a registration of the first device relative to the second device.

5. The computer-implemented method of claim 1 , further comprising:
in response to receiving additional selections of the first location and additional selections of the second location from additional devices, providing the reference frame to the additional devices to establish co-presence in the session, establishing the co-presence including generating:
a registration of the additional devices relative to the first device; and
a registration of the additional devices relative to the second device.

6. The computer-implemented method of claim 1 , wherein receiving, from the second device, the second selection of the first location and the second selection of the second location is based on displaying guidance, within a viewing window of the second device, the guidance including the first reference marker and the second reference marker.

7. The computer-implemented method of claim 1 , wherein the established co-presence is used to access the session and a game state is stored with the reference frame; and
re-establishing the reference frame includes the first device, the second device, or another device selecting upon the first location and the second location to gain access to the game according to the stored game state.

8. A computer program product tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to:
control a first device and a second device to detect at least one plane of a physical space associated with a session of a game for an augmented reality environment, the session provided as a service;
receive, from the first device, a first selection of a first location within an image capture of the physical space and a first selection of a second location within the image capture of the physical space;
generate a first reference marker corresponding to the first location and generating a second reference marker corresponding to the second location;
receive, from the second device, a second selection of the first location and a second selection of the second location;
generate a reference frame centered at the first reference marker that indicates a direction toward the second reference marker, the reference frame being generated using the at least one plane, the first location, and the second location; and
provide the reference frame to the first device and to the second device to establish co-presence in the session, wherein establishing the co-presence for the session includes generating a registration of the first device relative to the second device.

9. The computer program product of claim 8 , wherein establishing co-presence for the session further includes the second device calculating a transformation between a tracked location of the first device and a pose associated with the reference frame.

10. The computer program product of claim 8 , wherein the instructions further cause the at least one computing device to re-establish the session for a different physical space in response to detecting that at least the first device is located in a different physical space than the second device and the first device is requesting access to the session.

11. The computer program product of claim 10 , wherein re-establishing the session for the different physical space includes triggering initialization of an updated reference frame for the first device, the updated reference frame generated based at least in part on:
receiving, from the first device, selection of an updated first location and selection of an updated second location;
generating an updated first reference marker corresponding to the updated first location and generating an updated second reference marker corresponding to the updated second location;
correlating the first reference marker with the updated first reference marker; and
correlating the second reference marker with the updated second reference marker;
establishing the co-presence including generating, for the re-established session, a registration of the first device relative to the second device.

12. The computer program product of claim 8 , wherein receiving, from the second device, the second selection of the first location and the second selection of the second location is based on displaying guidance, within a viewing window of the second device, the guidance including the first reference marker and the second reference marker.

13. The computer program product of claim 8 , wherein the established co-presence is used to access the session and a game state is stored with the reference frame; and
re-establishing the reference frame includes the first device, the second device, or another device selecting upon the first location and the second location to gain access to the game according to the stored game state.

14. A system comprising:
a

Assignees

• Google Llc

Inventors

• Leeper Adam
• Ullman John
• Yang Cheng
• Tan Peter

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A63F13/65 Primary

  automatically by game devices or servers from real world data, e.g. measurement in live racing competition · CPC title

• G06V10/24

  Aligning, centring, orientation detection or correction of the image · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title

• A63F2300/8082

  Virtual reality · CPC title