Systems and methods for overlay of virtual object on proxy object

Patent metadata

Field	Value
Publication number	US-12505585-B2
Application number	US-202318124059-A
Country	US
Kind code	B2
Filing date	Mar 21, 2023
Priority date	Jan 20, 2023
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

To improve user experience when interacting with AR content within an AR environment, the AR content may be overlaid over a proxy object in a real-world space. However, issues such as the AR content and the proxy object not being aligned, or occlusion of the proxy object leading to glitching of the AR content, may decrease user enjoyment. In some embodiments, an AR system for the overlay of AR content may be implemented. The system may anchor a virtual model to the proxy object based on detected features on the proxy object. The anchoring may include aligning elements of the virtual model and proxy object. In response to an occluding object occluding some features on the proxy object, the system may anchor the virtual model to the occluding object, or to both the proxy object and the occluding object, based on detected features on the occluding object.

First claim

The invention claimed is:

1 . A computer-implemented method comprising:
detecting a first set of features on a physical object in a real-world space within a visual field of view; anchoring augmented reality (AR) content to the physical object using the detected first set of features; detecting a second set of features on an occluding object in the real-world space within the visual field of view; and responsive to the occluding object occluding one or more features of the detected first set of features, anchoring the AR content to both the occluding object using the detected second set of features and the physical object using one or more of the detected first set of features that is not occluded by the occluding object, wherein anchoring the AR content to both the occluding object and the physical object comprises:
aligning the AR content with the physical object by rendering the AR content overlaid over at least a portion of the physical object with an element of the AR content aligned with a respective element of the physical object; and
maintaining the aligning during movement of both the occluding object and the physical object;
and wherein the anchoring the AR content to both the occluding object and the physical object is further responsive to determining that the one or more of the detected first set of features that is not occluded by the occluding object and the detected second set of features on the occluding object are moving together.

2 . The computer-implemented method of claim 1 , wherein the element of the AR content is an axis of the AR content, and the respective element of the physical object is an axis of the physical object.

3 . The computer-implemented method of claim 1 , wherein the element of the AR content is a shape of at least a portion of the AR content, and the respective element of the physical object is a shape of at least a portion of the physical object.

4 . The computer-implemented method of claim 1 , wherein the determining that the one or more of the detected first set of features that is not occluded by the occluding object and the detected second set of features on the occluding object are moving together comprises detecting that a distance between a first feature of the one or more of the detected first set of features that is not occluded by the occluding object and a second feature of the detected second set of features on the occluding object is substantially constant.

5 . The computer-implemented method of claim 1 , wherein the anchoring the AR content to the occluding object is further responsive to determining that the occluding object is in contact with the physical object.

6 . The computer-implemented method of claim 1 , wherein the AR content is anchored to the physical object at a first alignment wherein a boundary of the AR content closest to the occluding object is not aligned with a respective boundary of the physical object closest to the occluding object, the method further comprising:
detecting that the occluding object is approaching the physical object from a particular direction; and responsive to the detecting that the occluding object is approaching the physical object from the particular direction, modifying the anchoring of the AR content to the physical object to a second alignment wherein the boundary of the AR content closest to the occluding object is aligned with the respective boundary of the physical object closest to the occluding object.

7 . The computer-implemented method of claim 1 further comprising:
prior to anchoring the AR content to the physical object, overlaying the AR content over at least a portion of the physical object;
maintaining the AR content at a fixed position while the physical object moves;
receiving an input indicating that the AR content is to anchor to the physical object; and
anchoring the AR content to the physical object, wherein the anchoring the AR content to the physical object occurs subsequent to receiving the input.

8 . The computer-implemented method of claim 1 further comprising:
responsive to the occluding object no longer occluding the one or more features of the detected first set of features, anchoring the AR content to the physical object using the detected first set of features.

9 . A system comprising:
at least one processor; and a memory storing processor-executable instructions that, when executed, cause the at least one processor to: detect a first set of features on a physical object in a real-world space within a visual field of view; anchor augmented reality (AR) content to the physical object using the detected first set of features; detect a second set of features on an occluding object in the real-world space within the visual field of view; and responsive to the occluding object occluding one or more features of the detected first set of features, anchor the AR content to both the occluding object using the detected second set of features and the physical object using one or more of the detected first set of features that is not occluded by the occluding object, wherein the at least processor is to anchor the AR content to both the occluding object and the physical object by performing operations including:
aligning the AR content with the physical object by rendering the AR content overlaid over at least a portion of the physical object with an element of the AR content aligned with a respective element of the physical object; and
maintaining the aligning during movement of both the occluding object and the physical object
and wherein the at least one processor is to anchor the AR content to both the occluding object and the physical object further responsive to determining that the one or more of the detected first set of features that is not occluded by the occluding object and the detected second set of features on the occluding object are moving together.

10 . The system of claim 3 , wherein the at least one processor is to determine that the one or more of the detected first set of features that is not occluded by the occluding object and the detected second set of features on the occluding object are moving together by performing operations including detecting that a distance between a first feature of the one or more of the detected first set of features that is not occluded by the occluding object and a second feature of the detected second set of features on the occluding object is substantially constant.

11 . The system of claim 9 , wherein the at least one processor is to anchor the AR content to the occluding object further responsive to determining that the occluding object is in contact with the physical object.

12 . The system of claim 9 , wherein the AR content is anchored to the physical object at a first alignment wherein a boundary of the AR content closest to the occluding object is not aligned with a respective boundary of the physical object closest to the occluding object, and the at least one processor is further to:
detect that the occluding object is approaching the physical object from a particular direction; and responsive to the detecting that the occluding object is approaching the physical object from the particular direction, modify the anchoring of the AR content to the physical object to a second alignment wherein the boundary of the AR content closest to the occluding object is aligned with the respective boundary of the physical object closest to the occluding object.

13 . The system of claim 9 , wherein the at least one processor is further to:
prior to anchoring the AR content to the physical object, overlay the AR content over at least a

Assignees

• Shopify Inc

Inventors

• Maschmeyer Russ
• Florenzano Eric Andrew
• Letkeman Brennan
• Bello Diego Macario
• Beauchamp Daniel

Classifications

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title