Systems and methods for overlay of virtual object on proxy object and concealment of proxy object

Patent metadata

Field	Value
Publication number	US-12417560-B2
Application number	US-202318124060-A
Country	US
Kind code	B2
Filing date	Mar 21, 2023
Priority date	Jan 20, 2023
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

To improve user experience when interacting with AR content within an AR environment, the AR content may be overlaid over a proxy object in a real-world space. Differences in dimension between the proxy object and the virtual model may be such that the object is larger than the virtual model, which may result in portions of the object appearing to protrude from behind the virtual model, decreasing user enjoyment. In some embodiments, an AR system for the overlay of AR content on a proxy object and concealment of the proxy object may be implemented. The system may overlay a virtual model to a proxy object, and then conceal any remaining visible portions of the proxy object from the visual field of a device displaying the AR environment. The system may overlay the virtual model so that any remaining visible portion of the proxy object is a single continuous region.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
overlaying augmented reality (AR) content onto a physical object in a real-world space within a visual field of view;
detecting a set of features on the physical object within the visual field of view;
anchoring the AR content to the physical object using at least one of the detected set of features;
responsive to detecting that a region of the physical object remains visible within the visual field of view subsequent to the overlaying of the AR content onto the physical object, concealing the region of the physical object within the visual field of view by visually altering a portion of the view corresponding to the region of the physical object; and
wherein anchoring the AR content to the physical object comprises:
aligning the AR content with the physical object by rendering the AR content with an element of the AR content aligned with a respective element of the physical object; and
maintaining the aligning during movement of the physical object so that the region of the physical object which remains visible within the visual field of view remains substantially the same during the movement.

2. The computer-implemented method of claim 1 wherein visually altering the portion of the view corresponding to the region of the physical object comprises visually altering at least some pixels corresponding to the region of the physical object, and wherein visually altering the at least some pixels comprises altering a respective pixel value of each pixel of the at least some pixels based on an area of the real-world space within the visual field of view outside of the region of the physical object that is not covered by the AR content.

3. The computer-implemented method of claim 1 , wherein detecting that the region of the physical object remains visible within the visual field of view comprises detecting that a subset of the detected set of features is not overlaid with the AR content.

4. The computer-implemented method of claim 1 , wherein the aligning includes the element of the AR content being aligned with the respective element of the physical object such that the region of the physical object which remains visible within the visual field of view is a single continuous region.

5. The computer-implemented method of claim 1 wherein the aligning the AR content with the physical object comprises:
determining a plurality of possible alignments between the AR content and the physical object, each of the plurality of possible alignments including one element of the AR content aligned with a respective one element of the physical object; and
selecting one of the plurality of possible alignments for the aligning the AR content with the physical object, wherein the selected one of the plurality of possible alignments has only a single continuous region as the region of the physical object which remains visible within the visual field of view.

6. A computer-implemented method comprising:
overlaying augmented reality (AR) content onto a physical object in a real-world space within a visual field of view;
responsive to detecting that a region of the physical object remains visible within the visual field of view subsequent to the overlaying of the AR content onto the physical object, concealing the region of the physical object within the visual field of view by visually altering a portion of the view corresponding to the region of the physical object; and
responsive to detecting that a portion of the region of the physical object which remains visible within the visual field of view has become occluded within the visual field of view, no longer concealing the portion that has become occluded.

7. The computer-implemented method of claim 6 wherein the portion of the region of the physical object becomes occluded within the visual field by an occluding object.

8. The computer-implemented method of claim 6 wherein the portion of the region of the physical object becomes occluded within the visual field by the AR content.

9. A system comprising:
at least one processor; and
a memory storing processor-executable instructions that, when executed, cause the at least one processor to:
overlay AR content onto a physical object in a real-world space within a visual field of view;
detect a set of features on the physical object within the visual field of view;
anchor the AR content to the physical object based on at least one of the detected set of features;
responsive to detecting that a region of the physical object remains visible within the visual field of view subsequent to the overlaying of the AR content onto the physical object, conceal the region of the physical object within the visual field of view by visually altering a portion of the view corresponding to the region of the physical object; and
wherein the at least one processor is to anchor the AR content to the physical object by performing operations including:
aligning the AR content with the physical object by rendering the AR content with an element of the AR content aligned with a respective element of the physical object; and
maintaining the aligning during movement of the physical object so that the region of the physical object which remains visible within the visual field of view remains substantially the same during the movement.

10. The system of claim 9 , wherein the at least one processor is to visually alter the portion of the view corresponding to the region of the physical object by performing operations including visually altering at least some pixels corresponding to the region of the physical object, and wherein the at least one processor is to visually alter the at least some pixels corresponding to the region of the physical object by performing operations including altering a respective pixel value of each pixel of the at least some pixels based on an area of the real-world space within the visual field of view outside of the region of the physical object that is not covered by the AR content.

11. The system of claim 9 , wherein the at least one processor is to detect the region of the physical object remains visible within the visual field of view by performing operations including detecting that a subset of the detected set of features is not overlaid with the AR content.

12. The system of claim 9 , wherein the aligning the AR content with the physical object includes the element of the AR content being aligned with the respective element of the physical object such that the region of the physical object which remains visible within the visual field of view is a single continuous region.

13. The system of claim 9 , wherein the at least one processor is to align the AR content with the physical object by performing operations including:
determining a plurality of possible alignments between the AR content and the physical object, each of the plurality of possible alignments including one element of the AR content aligned with a respective one element of the physical object; and
selecting one of the plurality of possible alignments for the aligning the AR content with the physical object, wherein the selected one of the plurality of possible alignments has only a single continuous region as the region of the physical object which remains visible within the visual field of view.

14. A system comprising:
at least one processor; and
a memory storing processor-executable instructions that, when executed, cause the at least one processor to:
overlay AR content onto a physical object in a real-world space within a visual field of view;
responsive to detecting that a region of the ph

Assignees

• Shopify Inc

Inventors

• Maschmeyer Russ
• Florenzano Eric Andrew
• Letkeman Brennan
• Bello Diego Macario
• Beauchamp Daniel

Classifications

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title