Systems and methods for providing augmented reality effects and three-dimensional mapping associated with interior spaces

Patent metadata

Field	Value
Publication number	US-2018190033-A1
Application number	US-201715849596-A
Country	US
Kind code	A1
Filing date	Dec 20, 2017
Priority date	Dec 30, 2016
Publication date	Jul 5, 2018
Grant date	—

Abstract

Systems, methods, and non-transitory computer readable media can obtain image data from a camera view associated with a computing device, where the image data is associated with an interior space. A portion of the image data for displaying one or more augmented reality (AR) content items can be determined. An AR content item to display in the camera view can be determined. The AR content item can be provided for presentation in the camera view based on the determined portion of the image data.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
obtaining, by a computing system, image data from a camera view associated with a computing device, the image data associated with an interior space; determining, by the computing system, a portion of the image data for displaying one or more augmented reality (AR) content items; determining, by the computing system, an AR content item to display in the camera view; and providing, by the computing system, the AR content item for presentation in the camera view based on the determined portion of the image data.

2 . The computer-implemented method of claim 1 , further comprising identifying one or more objects included in the image data from the camera view.

3 . The computer-implemented method of claim 2 , wherein the one or more objects are identified based on a machine learning model.

4 . The computer-implemented method of claim 2 , wherein the portion is an object identified in the image data from the camera view.

5 . The computer-implemented method of claim 1 , further comprising providing a three-dimensional (3D) map of the interior space.

6 . The computer-implemented method of claim 5 , wherein the 3D map is based on simultaneous localization and mapping (SLAM).

7 . The computer-implemented method of claim 5 , further comprising mapping the image data to the 3D map of the interior space in order to determine one or more of a location or a direction of a user associated with the camera view in the 3D map, wherein the camera view represents a perspective of the user.

8 . The computer-implemented method of claim 5 , wherein the 3D map includes a 3D model of one or more objects in the interior space.

9 . The computer-implemented method of claim 5 , wherein the portion is associated with one or more of a predefined section or a predefined object in the 3D map.

10 . The computer-implemented method of claim 1 , further comprising providing information in response to a gesture associated with the AR content item.

11 . A system comprising:
at least one hardware processor; and a memory storing instructions that, when executed by the at least one processor, cause the system to perform: obtaining image data from a camera view associated with a computing device, the image data associated with an interior space; determining a portion of the image data for displaying one or more augmented reality (AR) content items; determining an AR content item to display in the camera view; and providing the AR content item for presentation in the camera view based on the determined portion of the image data.

12 . The system of claim 11 , wherein the instructions further cause the system to perform identifying one or more objects included in the image data from the camera view.

13 . The system of claim 12 , wherein the portion is an object identified in the image data from the camera view.

14 . The system of claim 11 , wherein the instructions further cause the system to perform providing a three-dimensional (3D) map of the interior space.

15 . The system of claim 14 , wherein the portion is associated with one or more of a predefined section or a predefined object in the 3D map.

16 . A non-transitory computer readable medium including instructions that, when executed by at least one hardware processor of a computing system, cause the computing system to perform a method comprising:
obtaining image data from a camera view associated with a computing device, the image data associated with an interior space; determining a portion of the image data for displaying one or more augmented reality (AR) content items; determining an AR content item to display in the camera view; and providing the AR content item for presentation in the camera view based on the determined portion of the image data.

17 . The non-transitory computer readable medium of claim 16 , wherein the method further comprises identifying one or more objects included in the image data from the camera view.

18 . The non-transitory computer readable medium of claim 17 , wherein the portion is an object identified in the image data from the camera view.

19 . The non-transitory computer readable medium of claim 16 , wherein the method further comprises providing a three-dimensional (3D) map of the interior space.

20 . The non-transitory computer readable medium of claim 19 , wherein the portion is associated with one or more of a predefined section or a predefined object in the 3D map.

Assignees

• Facebook Inc

Inventors

• Barnett John Samuel
• Davis Dantley
• Lu Congxi
• Morton Jonathan
• Vajda Peter
• Harris Joshua Charles

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T15/205

  Image-based rendering · CPC title