System and method for geometric warping correction in projection mapping

Patent metadata

Field	Value
Publication number	US-10403028-B2
Application number	US-201816056763-A
Country	US
Kind code	B2
Filing date	Aug 7, 2018
Priority date	Jul 21, 2016
Publication date	Sep 3, 2019
Grant date	Sep 3, 2019

Abstract

A system and method for geometric warping correction in projection mapping is provided. A lower resolution mesh is applied to A mesh model, at least in a region of the mesh model misaligned with a corresponding region of a real-world object. One or more points of the lower resolution mesh are moved. In response, one or more corresponding points of the mesh model are moved to increase alignment between the region of the mesh model and the corresponding region of the real-world object. An updated mesh model is stored in a memory. And one or more projectors are controlled to projection map images corresponding to the updated mesh model onto the real-world object.

First claim

What is claimed is:

1. A system comprising:
one or more projectors configured to projection map onto a real-world object;
a memory storing a mesh model of the real-world object and images that are to be projection mapped onto the real-world object, the mesh model being preconfigured at the memory; and,
a controller configured to:
apply a lower resolution mesh to the mesh model, at least in a region of the mesh model misaligned with a corresponding region of the real-world object, the lower resolution mesh having one or more of a lower resolution and a lower point density than the mesh model, wherein the region of the mesh model misaligned with the corresponding region of the real-world object is determined by: comparing the mesh model with a point cloud representing the real-world object, the point cloud obtained independently from the mesh model, and wherein the lower resolution mesh comprises one or more of: a vertex clustering simplification of the mesh model; an incremental decimation of the mesh model, and, a subset of the high-res points of the mesh model;
move one or more low-res points of the lower resolution mesh; and, in response, move one or more corresponding high-res points of the mesh model, including any high-res points located between the one or more low-res points, to increase alignment between the region of the mesh model and the corresponding region of the real-world object;
store an updated mesh model in the memory based on the one or more corresponding high-res points of the mesh model that moved with the one or more low-res points of the lower resolution mesh; and,
control the one or more projectors to projection map the images corresponding to the updated mesh model onto the real-world object, the updated mesh model used in a geometric warping correction in the projection mapping, to warp the images for projection onto the real-world object.

2. The system of claim 1 , further comprising an input device, wherein the controller is further configured to determine the regions of the mesh model misaligned with the corresponding regions of the real-world object by: receiving input from the input device identifying the regions.

3. The system of claim 1 , wherein the lower resolution mesh is generated manually via input selecting sets of high-res points of the mesh model to be affected by a single newly created low res-point at a center of gravity of selected high-res points.

4. The system of claim 1 , wherein the controller is further configured to determine the region of the mesh model misaligned with the corresponding region of the real-world object by: comparing one or more of the mesh model and the lower resolution mesh with a point cloud representing the real-world object using one or more of point-comparison techniques, feature matching techniques, key-feature-related techniques and normal-related techniques.

5. The system of claim 1 , wherein the controller is further configured to automatically determine the region of the mesh model misaligned with the corresponding region of the real-world object by one or more of: calculating a respective distance of each of the one or more low-res points of the lower resolution mesh to a respective closest point in a point cloud representing the real-world object; and calculating a respective distance of each of the one or more high-res points of the mesh model to a respective closest point in a point cloud representing the real-world object.

6. The system of claim 1 , wherein the controller is further configured to automatically move one or more low-res points of the lower resolution mesh to an intersection of a respective lines determined from respective normals of respective vertexes of the lower resolution mesh and a mesh generated from a point cloud representing the real-world object until an alignment occurs between the region of the mesh model and a respective corresponding region of the point cloud.

7. The system of claim 1 , wherein the controller is further configured to move one or more low-res points of the lower resolution mesh until an alignment occurs between the region of the mesh model and a respective corresponding region of a point cloud representing the real-world object.

8. The system of claim 1 , wherein the controller is further configured to control the one or more projectors to projection map respective images corresponding to one or more intermediate updated mesh models onto the real-world object while the one or more low-res points of the lower resolution mesh are being moved.

9. A method comprising:
at a system comprising: one or more projectors configured to projection map onto a real-world object; a memory storing a mesh model of the real-world object and images that are to be projection mapped onto the real-world object, the mesh model being preconfigured at the memory; and, a controller, applying, using the controller, a lower resolution mesh to the mesh model, at least in a region of the mesh model misaligned with a corresponding region of the real-world object, the lower resolution mesh having one or more of a lower resolution and a lower point density than the mesh model), wherein the region of the mesh model misaligned with the corresponding region of the real-world object is determined by: comparing the mesh model with a point cloud representing the real-world object, the point cloud obtained independently from the mesh model, and wherein the lower resolution mesh comprises one or more of: a vertex clustering simplification of the mesh model; an incremental decimation of the mesh model, and a subset of the high-res points of the mesh model;
moving, using the controller, one or more low-res points of the lower resolution mesh; and, in response, move one or more corresponding high-res points of the mesh model, including any high-res points located between the one or more low-res points, to increase alignment between the region of the mesh model and the corresponding region of the real-world object;
storing, using the controller, an updated mesh model in the memory based on the one or more corresponding high-res points of the mesh model that moved with the one or more low-res points of the lower resolution mesh; and,
controlling, using the controller, the one or more projectors to projection map the images corresponding to the updated mesh model onto the real-world object, the updated mesh model used in a geometric warping correction in the projection mapping, to warp the images for projection onto the real-world object.

10. The method of claim 9 , wherein the system further comprises an input device, and the method further comprises determining the regions of the mesh model misaligned with the corresponding regions of the real-world object by: receiving input from the input device identifying the regions.

11. The method of claim 9 , further comprising generating the lower resolution mesh manually via input selecting sets of high-res points of the mesh model to be affected by a single newly created low res-point at a center of gravity of selected high-res points.

12. The method of claim 9 , further comprising determining the region of the mesh model misaligned with the corresponding region of the real-world object by one or more of: comparing one or more of the mesh model and the lower resolution mesh with a point cloud representing the real-world object using one or more of point-comparison techniques, feature matching techniques, key-feature-related techniques and normal-related techniques.

13. The method of claim 9 , further comprising automatically determining the region of the mesh model misaligned with the corresponding region of the real-world object by one or mor

Assignees

• Christie Digital Systems Usa

Inventors

• Anthony Roy
• Moule Kevin
• Scott Derek
• Wasilka Nick
• Elendt Maxwell

Classifications

• G06T2219/2021

  Shape modification · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T2210/36

  Level of detail · CPC title

• G06T15/10 Primary

  Geometric effects · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title