Systems and methods for providing augmented reality personalized content

Patent metadata

Field	Value
Publication number	US-11210854-B2
Application number	US-201715849620-A
Country	US
Kind code	B2
Filing date	Dec 20, 2017
Priority date	Dec 30, 2016
Publication date	Dec 28, 2021
Grant date	Dec 28, 2021

Abstract

Systems, methods, and non-transitory computer readable media can determine a placement in a camera view for displaying an augmented reality (AR) advertisement, where the camera view is associated with a computing device. An AR advertisement for a user associated with the computing device can be determined based on attributes associated with the user. Display of the AR advertisement can be caused at the determined placement in the camera view.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, by a computing system, a three dimensional map of an interior space associated with a camera view of a computing device;
determining, by the computing system, a location and a direction of a user associated with the computing device in the interior space based on a perspective represented by the camera view;
determining, by the computing system, a placement in the interior space for displaying an augmented reality (AR) advertisement based on the location and the direction of the user, wherein the determining the placement in the interior space comprises:
mapping image data of the camera view to the three dimensional map of the interior space, wherein the image data is uploaded to a system associated with the computing system;
identifying one or more objects reflected in the image data based on a machine learning model associated with the system, wherein the machine learning model is a classifier and trained based on images associated with the one or more objects; and
mapping the one or more objects to the three dimensional map of the interior space based on simultaneous localization and mapping (SLAM);

determining, by the computing system, the AR advertisement for the user based on attributes associated with the user, content created by the user in the system, content engaged with by the user in the system, and objects in the interior space with which the user previously interacted, wherein the attributes are obtained from a social graph associated with the user in the system; and
causing, by the computing system, display of the AR advertisement at the determined placement in the camera view.

2. The computer-implemented method of claim 1 , wherein the machine learning model is trained based on training data including at least one of objects or images associated with the objects.

3. The computer-implemented method of claim 1 , wherein the AR advertisement for the user is determined based at least in part on the one or more objects.

4. The computer-implemented method of claim 1 , wherein the one or more objects include an object in an exterior space in the camera view.

5. The computer-implemented method of claim 1 , wherein the placement is associated with a predefined section or a predefined object in the three dimensional map.

6. The computer-implemented method of claim 1 , wherein the attributes associated with the user include one or more of: an age, an age range, a gender, a geographical region, or an interest.

7. The computer-implemented method of claim 1 , wherein the system is a social networking system.

8. The computer-implemented method of claim 1 , wherein the AR advertisement provides information relating to the one or more objects identified in the camera view and includes information relating to at least one of products, stores, events, or product recommendations.

9. The computer-implemented method of claim 1 , wherein the AR advertisement includes a video overlaid on the one or more objects identified in the camera view and the video is selected by an organization associated with the interior space.

10. The computer-implemented method of claim 1 , wherein the AR advertisement includes a bot for assisting the user and the bot appears as at least one of an object or an animation within the camera view and provides messaging functionality for the user.

11. A computing system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
receiving a three dimensional map of an interior space associated with a camera view of a computing device;
determining a location and a direction of a user associated with the computing device in the interior space based on a perspective represented by the camera view;
determining a placement in the interior space for displaying an augmented reality (AR) advertisement based on the location and the direction of the user, wherein the determining the placement in the interior space comprises:
mapping image data of the camera view to the three dimensional map of the interior space, wherein the image data is uploaded to a system associated with the computing system;
identifying one or more objects reflected in the image data based on a machine learning model associated with the system, wherein the machine learning model is a classifier and trained based on images associated with the one or more objects; and
mapping the one or more objects to the three dimensional map of the interior space based on simultaneous localization and mapping (SLAM);

determining the AR advertisement for the user based on attributes associated with the user, content created by the user in the system, content engaged with by the user in the system, and objects in the interior space with which the user previously interacted, wherein the attributes are obtained from a social graph associated with the user in the system; and
causing display of the AR advertisement at the determined placement in the camera view.

12. The computing system of claim 11 , wherein the AR advertisement for the user is determined based at least in part on the one or more objects.

13. The computing system of claim 11 , wherein the placement is associated with a predefined section or a predefined object in the three dimensional map.

14. The computing system of claim 11 , wherein the AR advertisement provides information relating to the one or more objects identified in the camera view and includes information relating to at least one of products, stores, events, or product recommendations.

15. The computing system of claim 11 , wherein the AR advertisement includes a video overlaid on the one or more objects identified in the camera view and the video is selected by an organization associated with the interior space.

16. The computing system of claim 11 , wherein the AR advertisement includes a bot for assisting the user and the bot appears as at least one of an object or an animation within the camera view and provides messaging functionality for the user.

17. A non-transitory computer readable medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
receiving a three dimensional map of an interior space associated with a camera view of a computing device;
determining a location and a direction of a user associated with the computing device in the interior space based on a perspective represented by the camera view;
determining a placement in the interior space for displaying an augmented reality (AR) advertisement based on the location and the direction of the user, wherein the determining the placement in the interior space comprises:
mapping image data of the camera view to the three dimensional map of the interior space, wherein the image data is uploaded to a system associated with the computing system;
identifying one or more objects reflected in the image data based on a machine learning model associated with the system, wherein the machine learning model is a classifier and trained based on images associated with the one or more objects; and
mapping the one or more objects to the three dimensional map of the interior space based on simultaneous localization and mapping (SLAM);

determining the AR advertisement for the user based on attributes associated with the user, content created by the user in the system, content engaged with by the user in the system, and objects in the interior space with which the

Assignees

• Facebook Inc

Inventors

• Barnett John Samuel
• Davis Dantley
• Lu Congxi
• Morton Jonathan
• Vajda Peter
• Harris Joshua Charles

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V20/20

  in augmented reality scenes · CPC title