Resilient dynamic projection mapping system and methods

Patent metadata

Field	Value
Publication number	US-11270443-B2
Application number	US-201916727545-A
Country	US
Kind code	B2
Filing date	Dec 26, 2019
Priority date	Dec 26, 2018
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

Systems and methods for dynamically tracking objects, and projecting rendered 3D content onto said objects in real-time. The methods described herein further include image data capture performed by various image-capturing devices, wherein said data is segmented into various components to identify one or more projectors for rendering and projecting 3D content components onto one or more objects.

First claim

What is claimed is:

1. A method comprising:
obtaining, via one or more image capturing devices, image data associated with a first object;
generating, based on the image data, one or more point clouds associated with the first object;
for one or more contours of the first object:
determining, based on the one or more point clouds, at least one of position information and angle information for a contour of the first object;

dynamically rendering, based on the position information or angle information, a three-dimensional representation of the first object; and
using a plurality of projection devices to output for display the three-dimensional representation of the first object, wherein the three-dimensional representation is aligned with and projected onto the first object.

2. The method of claim 1 , further comprising:
using the one or more point clouds to generate a two-dimensional representation of a tracking space comprising the first object.

3. The method of claim 2 , wherein using the plurality of projection devices to output for display the three-dimensional representation of the first object further comprises:
determining that the first object is within the tracking space for a threshold period of time.

4. The method of claim 2 , wherein using the plurality of projection devices to output for display the three-dimensional representation of the first object further comprises:
determining that the first object has been monitored by the one or more image capturing devices for at least a threshold period of time.

5. The method of claim 1 , wherein dynamically rendering the three-dimensional representation of the first object further comprises:
sending, to a rendering engine, the position information or the angle information for the one or more contours of the first object.

6. The method of claim 1 , wherein determining at least one of position information or angle information for the contour of the first object further comprises:
monitoring, via the one or more image capturing devices and during a first time period, the position information or the angle information for the contour of the first object.

7. The method of claim 1 , wherein the three-dimensional representation of the first object comprises a footwear design.

8. The method of claim 1 , further comprising:
determining, based on a stored three-dimensional model, a first set of contours;
determining position information and angle information for each contour in the first set of contours; and
comparing, based on respective position information and angle information, the one or more contours of the first object with the first set of contours.

9. The method of claim 1 , wherein outputting for display the three-dimensional representation of the first object, further comprises:
outputting for display, via the plurality of projection devices and onto the first object, one or more surface regions associated with the three-dimensional representation of the first object.

10. An apparatus comprising:
one or more processors; and
memory operatively coupled to the one or more processors and storing computer readable instructions that, when executed, cause the apparatus to:
obtain, via one or more image capturing devices, image data associated with a first object;
generate, based on the image data, one or more point clouds associated with the first object;
for one or more contours of the first object:
determine, based on the one or more point clouds, at least one of position information and angle information for a contour of the first object;

dynamically render, based on the position information or angle information, a three-dimensional representation of the first object; and
use a plurality of projection devices to output for display the three-dimensional representation of the first object, wherein the three-dimensional representation is aligned with and projected onto the first object.

11. The apparatus of claim 10 , wherein the computer readable instructions, when executed, further cause the apparatus to:
use the one or more point clouds to generate a two-dimensional representation of a tracking space comprising the first object.

12. The apparatus of claim 11 , wherein the computer readable instructions, when executed, further cause the apparatus to use the plurality of projection devices to output for display the three-dimensional representation of the first object by:
determining that the first object is within the tracking space for a threshold period of time.

13. The apparatus of claim 11 , wherein the computer readable instructions, when executed, further cause the apparatus to use the plurality of projection devices to output for display the three-dimensional representation of the first object by:
determining that the first object has been monitored by the one or more image capturing devices for at least a threshold period of time.

14. The apparatus of claim 10 , wherein the computer readable instructions, when executed, further cause the apparatus to dynamically render the three-dimensional representation of the first object by:
sending, to a rendering engine, the position information or the angle information for the one or more contours of the first object.

15. The apparatus of claim 10 , wherein the computer readable instructions, when executed, further cause the apparatus to determine at least one of position information or angle information for the contour of the first object by:
monitoring, via the one or more image capturing devices and during a first time period, the position information or the angle information for the contour of the first object.

16. The apparatus of claim 10 , wherein the three-dimensional representation of the first object comprises a footwear design.

17. The apparatus of claim 10 , wherein the computer readable instructions, when executed, further cause the apparatus to:
determine, based on a stored three-dimensional model, a first set of pre-determined contours;
determine position information and angle information for each of the first set of pre-determined contours; and
compare, based on respective position information and angle information, the one or more contours of the first object with the first set of pre-determined contours.

18. A system comprising:
a first computing device;
one or more image capturing devices; and
a plurality of projection devices;
wherein the first computing device comprises:
one or more first processors; and
memory storing first computer readable instructions that, when executed by the one or more first processors, cause the first computing device to:
obtain, via one or more image capturing devices, image data associated with a first object;
generate, based on the image data, one or more point clouds associated with the first object;
for one or more contours of the first object:
determine, based on the one or more point clouds, at least one of position information and angle information for a contour of the first object;

dynamically render, based on the position information or angle information, a three-dimensional representation of the first object; and
use a plurality of projection devices to output for display the three-dimensional representation of the first object, wherein the three-dimensional representation is aligned with and projected onto the first object.

19. The system of claim 18 , wherein the first computer readable instructions, when executed by the one or more first processors, further cause the first computing device to:
use the

Assignees

• Nike Inc

Inventors

• Legaye Robert G
• Coyle Ian
• Leblanc Ian
• Purdy Ben
• Wester Thomas

Classifications

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/251 Primary

  involving models · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G01B11/2504

  Calibration devices · CPC title