Resilient dynamic projection mapping system and methods

Patent metadata

Field	Value
Publication number	US-11816848-B2
Application number	US-202217590984-A
Country	US
Kind code	B2
Filing date	Feb 2, 2022
Priority date	Dec 26, 2018
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

Systems and methods for dynamically tracking objects, and projecting rendered 3D content onto said objects in real-time. The methods described herein further include image data capture performed by various image-capturing devices, wherein said data is segmented into various components to identify one or more projectors for rendering and projecting 3D content components onto one or more objects.

First claim

What is claimed is:

1. A method comprising:
obtaining image data associated with an object;
for one or more contours of the object:
determining position information or angle information for a contour of the object;

rendering, based on the position information or angle information, a three-dimensional representation of the object; and
using a plurality of projection devices to output for display the three-dimensional representation of the object, wherein the three-dimensional representation is aligned with and projected onto the object.

2. The method of claim 1 , further comprising:
using one or more point clouds to generate a two-dimensional representation of a tracking space comprising the object.

3. The method of claim 2 , wherein using the plurality of projection devices to output for display the three-dimensional representation of the object further comprises:
determining that first object is within the tracking space for a threshold period of time.

4. The method of claim 2 , wherein using the plurality of projection devices to output for display the three-dimensional representation of the object further comprises:
determining that the object has been monitored by an image capturing device for at least a threshold period of time.

5. The method of claim 1 , wherein rendering the three-dimensional representation of the object further comprises:
sending, to a rendering engine, the position information or the angle information for the one or more contours of the object.

6. The method of claim 1 , wherein determining the position information or the angle information for the contour of the object further comprises:
monitoring, via one or more image capturing devices and during a first time period, the position information or the angle information for the contour of the object.

7. The method of claim 1 , wherein the three-dimensional representation of the object comprises a footwear design.

8. The method of claim 1 , further comprising:
determining, based on a stored three-dimensional model, a first set of contours;
determining position information and angle information for each contour in the first set of contours; and
comparing, based on respective position information and angle information, the one or more contours of the object with the first set of contours.

9. The method of claim 1 , wherein outputting for display the three-dimensional representation of the object, further comprises:
outputting for display, via the plurality of projection devices and onto the object, one or more surface regions associated with the three-dimensional representation of the object.

10. An apparatus comprising:
one or more processors; and
memory operatively coupled to the one or more processors and storing instructions that, when executed, cause the apparatus to:
obtain image data associated with an object;
for one or more contours of the object:
determine position information or angle information for a contour of the object;

render, based on the position information or angle information, a three-dimensional representation of the object; and
use a plurality of projection devices to output for display the three-dimensional representation of the object, wherein the three-dimensional representation is aligned with and projected onto the object.

11. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to:
use one or more point clouds to generate a two-dimensional representation of a tracking space comprising the object.

12. The apparatus of claim 11 , wherein the instructions, when executed, further cause the apparatus to use the plurality of projection devices to output for display the three-dimensional representation of the object by:
determining that the object is within the tracking space for a threshold period of time.

13. The apparatus of claim 11 , wherein the instructions, when executed, further cause the apparatus to use the plurality of projection devices to output for display the three-dimensional representation of the object by:
determining that the object has been monitored by an image capturing device for at least a threshold period of time.

14. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to render the three-dimensional representation of the object by:
sending, to a rendering engine, the position information or the angle information for the one or more contours of the object.

15. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to determine the position information or the angle information for the contour of the object by:
monitoring, via one or more image capturing devices and during a first time period, the position information or the angle information for the contour of the object.

16. The apparatus of claim 10 , wherein the three-dimensional representation of the object comprises a footwear design.

17. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to:
determine, based on a stored three-dimensional model, a first set of contours;
determine position information and angle information for each contour in the first set of contours; and
compare, based on respective position information and angle information, the one or more contours of the object with the first set of contours.

18. The apparatus of claim 10 , wherein the instructions, when executed, further cause the apparatus to output for display the three-dimensional representation of the object by:
outputting for display, via the plurality of projection devices and onto the object, one or more surface regions associated with the three-dimensional representation of the object.

19. One or more non-transitory computer readable media storing instructions that, when executed cause:
obtaining image data associated with an object;
for one or more contours of the object:
determining position information or angle information for a contour of the object;

rendering, based on the position information or angle information, a three-dimensional representation of the object; and
using a plurality of projection devices to output for display the three-dimensional representation of the object, wherein the three-dimensional representation is aligned with and projected onto the object.

20. The one or more non-transitory computer readable media of claim 19 , wherein the instructions, when executed, further cause:
using one or more point clouds to generate a two-dimensional representation of a tracking space comprising the object.

21. The one or more non-transitory computer readable media of claim 20 , wherein the instructions, when executed, further cause using the plurality of projection devices to output for display the three-dimensional representation of the object by:
determining that first object is within the tracking space for a threshold period of time.

22. The one or more non-transitory computer readable media of claim 20 , wherein the instructions, when executed, further cause using the plurality of projection devices to output for display the three-dimensional representation of the object by:
determining that the object has been monitored by an image capturing device for at least a threshold period of time.

23. The one or more non-transitory computer readable media of claim 19 , wherein the instructions, when executed, further cause rendering the three-dimensio

Assignees

• Nike Inc

Inventors

• Legaye Robert G
• Coyle Ian
• Leblanc Ian
• Purdy Ben
• Wester Thomas

Classifications

• G06T7/251 Primary

  involving models · CPC title

• G01B11/2504

  Calibration devices · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title