Multi-device mapping and collaboration in augmented-reality environments

Patent metadata

Field	Value
Publication number	US-10854004-B2
Application number	US-201816112475-A
Country	US
Kind code	B2
Filing date	Aug 24, 2018
Priority date	Aug 24, 2018
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

In one embodiment, a method includes receiving first information from a sensor associated with a first computing device, wherein the first information comprises information associated with first images captured at the first sensor; receiving second information from a second computing device, wherein the second information comprises information associated with second images captured at a sensor associated with the second computing device; identifying first points within the first images; identifying second points within the second images; and relocalizing the first and second computing devices within a shared augmented-reality environment by defining coordinate spaces based on the images and combining the coordinate spaces based on identified shared points.

First claim

What is claimed is:

1. A method comprising:
by a processor of a first computing device, receiving first information from a first sensor associated with the first computing device, wherein the first information comprises information associated with one or more first images captured at the first sensor;
receiving second information from a second computing device, wherein the second information comprises information associated with one or more second images captured at a second sensor associated with the second computing device;
identifying one or more first points within the first images;
identifying one or more second points within the second images;
defining a first coordinate space comprising the first points, wherein each of the first points is associated with a coordinate within the first coordinate space based on the first information;
defining a second coordinate space comprising the second points, wherein each of the second points is associated with a coordinate within the second coordinate space based on the second information;
identifying one or more shared points, wherein each shared point is a point at which a particular first point and a particular second point intersect;
combining the first coordinate space with the second coordinate space to create a shared coordinate space corresponding to a shared augmented-reality environment based on the identified shared points; and
updating a master coordinate space corresponding to the shared augmented-reality environment by incorporating information corresponding to each identified shared point associated with a number of perspectives greater than a number of perspectives associated with a corresponding point of the master coordinate space.

2. The method of claim 1 , further comprising:
receiving content information from the second computing device, wherein the content information comprises information defining one or more locations in the shared augmented-reality environment and information specifying an augmented-reality content item;
placing the augmented-reality content item within the shared augmented-reality environment at a location associated with the defined locations; and
rendering the shared augmented-reality environment and the augmented-reality content item on a display associated with the first device.

3. The method of claim 2 , wherein the second computing device further comprises an inertial measurement unit and a processor, and wherein the second sensor is a camera.

4. The method of claim 3 , wherein the second computing device is a stylus device.

5. The method of claim 2 , wherein the augmented-reality content item comprises a portion of a drawing, wherein the drawing is constructed based on a gesture input received from the second computing device.

6. The method of claim 2 , wherein the augmented-reality content item comprises a photo, a video, an image file, or a virtual object.

7. The method of claim 2 , wherein the augmented-reality content item comprises:
a mask that is configured to be overlaid on an object identified in the shared augmented-reality environment, or
a filter overlay that is configured to be overlaid on a rendering of the shared augmented-reality environment.

8. The method of claim 2 , wherein the augmented-reality content item comprises a visual representation of a communication on an online social network.

9. The method of claim 1 , wherein the first sensor comprises a camera, a LiDAR sensor, or a radar sensor.

10. The method of claim 1 , further comprising:
receiving information defining the shared augmented-reality environment;
receiving update information from the second computing device, wherein the update information defines one or more additional second points; and
updating the shared augmented-reality environment based on the additional second points.

11. The method of claim 10 , wherein the additional second points are received periodically.

12. The method of claim 10 , wherein the additional second points are received when a threshold number of additional second points is identified.

13. The method of claim 1 , further comprising:
rendering the shared augmented-reality environment on the first computing device;
overlaying the rendering of the shared augmented-reality environment with a matrix of visual markers, wherein:
each visual marker corresponds to a respective coordinate within the augmented-reality environment, and
each visual marker is configured to vary in appearance based on a distance between a display associated with the first computing device and a location of the visual marker within the shared augmented-reality environment.

14. The method of claim 1 , wherein the first information is received at a first time-point and the second information is received at a second time-point.

15. The method of claim 1 , further comprising:
relocalizing the first computing device and the second computing device within the shared augmented-reality environment based on the identified shared points of the shared coordinate space.

16. The method of claim 15 , wherein the first computing device is associated with a first user and the second computing device is associated with a second user, further comprising:
accessing a social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between them, the nodes comprising:
a first node corresponding to the first user, and
a second node corresponding to the second user;

determining a presence of at least one edge that directly connects the first node and the second node; and
in response to determining the presence of the at least one edge, authorizing the relocalizing of the first computing device and the second computing device.

17. The method of claim 15 , wherein the first computing device is associated with a first user and the second computing device is associated with a second user, further comprising:
determining that the first user and the second user are attending a common event at a first site, wherein the first site is associated with the shared augmented-reality environment; and
in response to determining that the first user and the second user are attending the common event at the first site, authorizing the relocalizing of the first computing device and the second computing device.

18. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
by a processor of a first computing device, receive first information from a first sensor associated with the first computing device, wherein the first information comprises information associated with one or more first images captured at the first sensor;
receive second information from a second computing device, wherein the second information comprises information associated with one or more second images captured at a second sensor associated with the second computing device;
identify one or more first points within the first images;
identify one or more second points within the second images;
define a first coordinate space comprising the first points, wherein each of the first points is associated with a coordinate within the first coordinate space based on the first information;
define a second coordinate space comprising the second points, wherein each of the second points is associated with a coordinate within the second coordinate space based on the second information;
identify one or more shared points, wherein each shared point is a poin

Assignees

• Facebook Inc

Inventors

• Stahl Justin David
• Sead Narin
• Murrell Tullie
• Germe Gregory Daniel Louis
• Kish Steven

Classifications

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V40/173

  face re-identification, e.g. recognising unknown faces across different face tracks · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title