Occlusion detection system

Patent metadata

Field	Value
Publication number	US-11983307-B2
Application number	US-202318119079-A
Country	US
Kind code	B2
Filing date	Mar 8, 2023
Priority date	Sep 10, 2019
Publication date	May 14, 2024
Grant date	May 14, 2024

Abstract

An occlusion detection system to perform operations that include: capturing image data that depicts an environment at a client device, the environment including a target object at a position within the environment; causing display of a presentation of the environment at the client device, the presentation of the environment including a display of the target object at the position within the environment; detecting a first attribute of the display of the target object at the client device; performing a comparison of the first attribute of the display of the target object and a second attribute associated with the target object; and detecting an occlusion based on the comparison.

First claim

What is claimed is:

1. A method comprising:
accessing image data from a client device, the image data comprising a set of image features that depict a display of an object, the object comprising a property;
determining a perspective of the client device relative to the object based on the set of image features;
accessing a repository that includes a set of expected properties associated with the object based on the perspective of the client device;
detecting an occlusion based on a comparison of the set of expected properties and the property of the object;
determining an attribute of the occlusion; and
causing display of media content at a position within the image data based on the attribute of the occlusion.

2. The method of claim 1 , wherein the set of expected properties include pixel color values.

3. The method of claim 1 , wherein the attribute of the occlusion includes a semantic feature of the occlusion.

4. The method of claim 1 , wherein the media content comprises augmented-reality content.

5. The method of claim 1 , further comprising:
determining a location of the client device;
identifying the object based on the location of the client device; and
accessing the repository that includes the set of expected properties associated with the object based on the perspective of the client device and the identifying the object.

6. The method of claim 1 , wherein the determining the location of the client device includes detecting the client device within a geo-fence.

7. A system comprising:
a memory; and
at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising:
accessing image data from a client device, the image data comprising a set of image features that depict a display of an object, the object comprising a property;
determining a perspective of the client device relative to the object based on the set of image features;
accessing a repository that includes a set of expected properties associated with the object based on the perspective of the client device;
detecting an occlusion based on a comparison of the set of expected properties and the property of the object;
determining an attribute of the occlusion; and
causing display of media content at a position within the image data based on the attribute of the occlusion.

8. The system of claim 7 , wherein the set of expected properties include pixel color values.

9. The system of claim 7 , wherein the attribute of the occlusion includes a semantic feature of the occlusion.

10. The system of claim 7 , wherein the media content comprises augmented-reality content.

11. The system of claim 7 , further comprising:
determining a location of the client device;
identifying the object based on the location of the client device; and
accessing the repository that includes the set of expected properties associated with the object based on the perspective of the client device and the identifying the object.

12. The system of claim 7 , wherein the determining the location of the client device includes detecting the client device within a geo-fence.

13. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
accessing image data from a client device, the image data comprising a set of image features that depict a display of an object, the object comprising a property;
determining a perspective of the client device relative to the object based on the set of image features;
accessing a repository that includes a set of expected properties associated with the object based on the perspective of the client device;
detecting an occlusion based on a comparison of the set of expected properties and the property of the object;
determining an attribute of the occlusion; and
causing display of media content at a position within the image data based on the attribute of the occlusion.

14. The non-transitory machine-readable storage medium of claim 13 , wherein the set of expected properties include pixel color values.

15. The non-transitory machine-readable storage medium of claim 13 , wherein the attribute of the occlusion includes a semantic feature of the occlusion.

16. The non-transitory machine-readable storage medium of claim 13 , wherein the media content comprises augmented-reality content.

17. The non-transitory machine-readable storage medium of claim 13 , further comprising:
determining a location of the client device;
identifying the object based on the location of the client device; and
accessing the repository that includes the set of expected properties associated with the object based on the perspective of the client device and the identifying the object.

Assignees

• Snap Inc

Inventors

• Cowburn Piers
• Li David
• Müller Sandvik Isac Andreas
• Pan Qi

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06V20/20

  in augmented reality scenes · CPC title