Crowd sourced mapping system

Patent metadata

Field	Value
Publication number	US-11798243-B2
Application number	US-202217948813-A
Country	US
Kind code	B2
Filing date	Sep 20, 2022
Priority date	Jun 20, 2019
Publication date	Oct 24, 2023
Grant date	Oct 24, 2023

Abstract

A crowd-sourced modeling system to perform operations that include: receiving image data that comprises image attributes; accessing a 3D model based on at least the image attributes of the image data, wherein the 3D model comprises a plurality of parts that collectively depict an object or environment; identifying a change in the object or environment based on a comparison of the image data with the plurality of parts of the 3D model, the change corresponding to a part of the 3D model from among the plurality of parts; and generating an update to the part of the 3D model based on the image attributes of the image data.

First claim

What is claimed is:

1. A method comprising:
receiving image data that comprises a set of image attributes that depict a location;
accessing a three-dimensional (3D) model that corresponds with an object located at the location responsive to the receiving the image data, the 3D model comprising a mesh representation of a surface of the location, wherein a portion of the mesh representation corresponds with a position of the object within the location;
detecting a change to the portion of the mesh representation that corresponds with the position of the object based on a comparison of the image attributes of the image data with the mesh representation of the surface of the location; and
generating an update to the portion of the mesh representation that corresponds with the position of the object within the location based on the image attributes of the image data responsive to the detecting the change.

2. The method of claim 1 , wherein the receiving the image data includes:
accessing a repository that comprises a collection of image data associated with the location, the collection of image data including at least the image data; and
receiving the image data from the repository.

3. The method of claim 1 , wherein the receiving the image data further comprises:
causing display of a request for the image data at the client device; and
receiving the image data at the client device.

4. The method of claim 3 , wherein the causing display of the request for the image data includes:
detecting the client device within a threshold distance of the location; and
causing display of the request for the image data in response to the detecting the client device within the threshold distance of the location.

5. The method of claim 3 , wherein the causing display of the request for the image data includes:
detecting a change in an environment that corresponds with the location; and
causing display of the request for the image data in response to the change.

6. The method of claim 1 , wherein the 3D model includes a 3D mesh model that depicts a surface feature of the location.

7. The method of claim 1 , wherein the receiving the image data includes:
selecting the image data based on the set of image attributes of the image data, the set of image attributes including image metadata.

8. A system comprising:
a memory; and
at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising:
receiving image data that comprises a set of image attributes that depict a location;
accessing a three-dimensional (3D) model that corresponds with an object located at the location responsive to the receiving the image data, the 3D model comprising a mesh representation of a surface of the location, wherein a portion of the mesh representation corresponds with a position of the object within the location;
detecting a change to the portion of the mesh representation that corresponds with the position of the object based on a comparison of the image attributes of the image data with the mesh representation of the surface of the location; and
generating an update to the portion of the mesh representation that corresponds with the position of the object within the location based on the image attributes of the image data responsive to the detecting the change.

9. The system of claim 8 , wherein the receiving the image data includes:
accessing a repository that comprises a collection of image data associated with the location, the collection of image data including at least the image data; and
receiving the image data from the repository.

10. The system of claim 8 , wherein the receiving the image data further comprises:
causing display of a request for the image data at the client device; and
receiving the image data at the client device.

11. The system of claim 10 , wherein the causing display of the request for the image data includes:
detecting the client device within a threshold distance of the location; and
causing display of the request for the image data in response to the detecting the client device within the threshold distance of the location.

12. The system of claim 10 , wherein the causing display of the request for the image data includes:
detecting a change in an environment that corresponds with the location; and
causing display of the request for the image data in response to the change.

13. The system of claim 8 , wherein the 3D model includes a 3D mesh model that depicts a surface feature of the location.

14. The system of claim 8 , wherein the receiving the image data includes:
selecting the image data based on the set of image attributes of the image data, the set of image attributes including image metadata.

15. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
receiving image data that comprises a set of image attributes that depict a location;
accessing a three-dimensional (3D) model that corresponds with an object located at the location responsive to the receiving the image data, the 3D model comprising a mesh representation of a surface of the location, wherein a portion of the mesh representation corresponds with a position of the object within the location;
detecting a change to the portion of the mesh representation that corresponds with the position of the object based on a comparison of the image attributes of the image data with the mesh representation of the surface of the location; and
generating an update to the portion of the mesh representation that corresponds with the position of the object within the location based on the image attributes of the image data responsive to the detecting the change.

16. The non-transitory machine-readable storage medium of claim 15 , wherein the receiving the image data includes:
accessing a repository that comprises a collection of image data associated with the location, the collection of image data including at least the image data; and
receiving the image data from the repository.

17. The non-transitory machine-readable storage medium of claim 15 , wherein the receiving the image data further comprises:
causing display of a request for the image data at the client device; and
receiving the image data at the client device.

18. The non-transitory machine-readable storage medium of claim 17 , wherein the causing display of the request for the image data includes:
detecting the client device within a threshold distance of the location; and
causing display of the request for the image data in response to the detecting the client device within the threshold distance of the location.

19. The non-transitory machine-readable storage medium of claim 17 , wherein the causing display of the request for the image data includes:
detecting a change in an environment that corresponds with the location; and
causing display of the request for the image data in response to the change.

20. The non-transitory machine-readable storage medium of claim 15 , wherein the 3D model includes a 3D mesh model that depicts a surface feature of the location.

Assignees

• Snap Inc

Inventors

• Cowburn Piers
• Müller Sandvik Isac Andreas
• Pan Qi
• Li David

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T17/05 Primary

  Geographic models · CPC title

• G06F16/51

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/54

  Browsing; Visualisation therefor · CPC title

• G06T7/97

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title