Projecting images and videos onto engineered curved surfaces

Patent metadata

Field	Value
Publication number	US-11082673-B2
Application number	US-201916575639-A
Country	US
Kind code	B2
Filing date	Sep 19, 2019
Priority date	Feb 6, 2018
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

Techniques for projecting images onto fabricated surfaces. One or more electronic files containing an engineering diagram that describes a three-dimensional shape of the fabricated surface are retrieved. Warp transformation information for transforming images to be projected onto the fabricated surface is calculated, based on the engineering diagram, positional information defining a three-dimensional position of a projection device, relative to the fabricated surface, and positional information of an intended viewer relative to the fabricated surface. The warp transformation information is applied to one or more images to generate one or more transformed images. Embodiments project, using the projection device, the one or more transformed images onto the fabricated surface.

First claim

What is claimed is:

1. A method, comprising:
receiving one or more electronic files containing an engineering diagram relating to a surface;
parsing the one or more electronic files to determine a plurality of parameters describing a three-dimensional shape of the surface;
receiving first positional information describing a location of a projection device in three-dimensional space;
calculating warp transformation information for transforming a plurality of images for projection onto the surface, based on the determined plurality of parameters, the first positional information, and second positional information of an intended viewer relative to the surface, and further based on mapping a first image of the plurality of images with a simulated projection of the first image onto a virtual representation of the surface;
receiving one or more images to project onto the surface;
applying the warp transformation information to the one or more images to generate one or more transformed images; and
projecting, using the projection device, the one or more transformed images onto the surface.

2. The method of claim 1 , wherein the one or more images further comprise frames of video data.

3. The method of claim 1 , wherein the simulated projection of the first image is based on the determined plurality of parameters and the first positional information defining the three-dimensional position of the projection device, relative to the surface.

4. The method of claim 1 , wherein mapping the first image of the plurality of images with the simulated projection further comprises:
mapping a plurality of points between the first image and the simulated projection of the first image.

5. The method of claim 1 , wherein the warp transformation information is applied to the one or more images to generate one or more transformed images in real-time using one or more graphics processing units (GPUs).

6. The method of claim 1 , wherein the one or more images further comprise frames of video data, and further comprising generating a digital file that is formatted according to a predefined video encoding standard, wherein the digital file contains at least a representation of the transformed images generated from the frames of video data.

7. The method of claim 1 , wherein the first positional information further defines a pose and an orientation of the projection device.

8. The method of claim 7 , wherein the warp transformation information is applied to the one or more images to generate one or more transformed images, such that when the one or more transformed images are projected onto the surface using the projection device, the projected image is substantially similar in appearance to the one or more images when viewed from a location of the intended viewer.

9. The method of claim 1 , wherein the projection device further comprises a normal-throw projection device, and wherein the first positional information further comprises three-dimensional coordinates defining a center of a visible lens of the normal-throw projection device.

10. The method of claim 1 , wherein the first positional information is calculated using multiple different projector positions.

11. The method of claim 1 , wherein calculating warp transformation information for transforming images to be projected onto the surface is further based on generating a three-dimensional virtual representation of the surface, using the engineering diagram.

12. The method of claim 11 , wherein calculating warp transformation information for transforming images to be projected onto the surface, based on the determined plurality of parameters and first positional information defining a three-dimensional position of the projection device, relative to the surface further comprises:
receiving a selection of a bounding box that corresponds to a rectangular portion of a camera plane; and
calculating the warp transformation information, based on the bounding box selection and one or more generated bijective mappings relating to a projector plane for the projection device and the camera plan.

13. A projection system, comprising:
one or more projection devices;
a non-transitory computer-readable medium containing electronic data corresponding to one or more images to project onto a surface and warp transformation information for transforming a plurality of images for projection onto the surface, wherein the warp transformation information is calculated based on a plurality of parameters describing a three-dimensional shape of the surface, first positional information describing one or more locations of the one or more projection devices in three-dimensional space, and second positional information of an intended viewer relative to the surface, and wherein the warp transformation information is further based on mapping a first image of the plurality of images with a simulated projection of the first image onto a virtual representation of the surface; and
logic configured to:
parse one or more electronic files containing an engineering diagram, to determine the plurality of parameters describing the three-dimensional shape of the surface
retrieve the electronic data corresponding to the one or more images to project onto the surface from the non-transitory computer-readable medium;
apply the warp transformation information to the one or more images to generate one or more transformed images; and
project, using the one or more projection devices, the one or more transformed images onto the surface.

14. The projection system of claim 13 , wherein the one or more images further comprise frames of video data.

15. The projection system of claim 13 , wherein the simulated projection of the first image is based on the determined plurality of parameters and the first positional information defining the three-dimensional position of the one or more projection devices, relative to the surface; and
wherein mapping the first image of the plurality of images with the simulated projection further comprises: mapping a plurality of points between the first image and the simulated projection of the first image.

16. The projection system of claim 13 , wherein the warp transformation information for transforming images to be projected onto the surface is further based on
generating a three-dimensional virtual representation of the surface, using the engineering diagram.

17. The projection system of claim 13 , wherein the first positional information further defines a pose and an orientation of the one or more projection devices, and wherein the second positional information defines a location of the intended viewer.

18. The projection system of claim 17 , wherein the warp transformation information is applied to the one or more images to generate one or more transformed images, such that when the one or more transformed images are projected onto the surface using the one or more projection devices, the projected image is substantially similar in appearance to the one or more images when viewed from the location of the intended viewer.

19. A non-transitory computer-readable medium containing computer program code that, when executed by operation of one or more computer processors, performs an operation comprising:
receiving one or more electronic files containing an engineering diagram relating to a surface;
parsing the one or more electronic files to determine a plurality of parameters describing a three-dimensional shape of the surface;
receiving first positional information describing a location of a projection devi

Assignees

• Boeing Co

Inventors

• Tripathi Shubham
• Viswanathan Kapaleeswaran
• Ayyagari Arun

Classifications

• B64D11/0015

  Arrangements for entertainment or communications, e.g. radio, television · CPC title

• G06T15/10

  Geometric effects · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• H04N9/3147

  Multi-projection systems (video walls G06F3/1446) · CPC title

• H04N9/3194

  including sensor feedback · CPC title