Systems and methods for augmenting panoramic image data with performance related data for a building

Patent metadata

Field	Value
Publication number	US-9330501-B2
Application number	US-201213590014-A
Country	US
Kind code	B2
Filing date	Aug 20, 2012
Priority date	Aug 20, 2012
Publication date	May 3, 2016
Grant date	May 3, 2016

Abstract

A system and method for augmenting panoramic images with performance related data for a building are disclosed. The system includes a memory coupled to a processor configured to receive panoramic image data, generate a digital image based on the panoramic image data, and modify the digital image to include a graphical representation of the performance related data. The processor maps performance related data to the coordinate system of the panoramic images such that the graphical representation may be generated without converting the panoramic image data into a high-quality, three-dimensional model in a CAD program. In this way, computer-generated graphical representations of building performance data, such as temperature fields or virtual structures, may be overlaid directly onto a two-dimensional projection of the panoramic image data. In one embodiment, the panoramic image data is a spherical, 360 degree panoramic image captured by a laser scanning device.

First claim

What is claimed is:

1. A method for augmenting panoramic images with performance related data for a building, the method comprising:
receiving panoramic image data for a spherical panoramic image, the panoramic image data comprising color information and metadata for each pixel in the spherical panoramic image;
generating a digital image comprising a two-dimensional (2D) projection plane based on at least a portion of the spherical panoramic image; and
modifying the digital image to include a graphical representation of performance related data based on the metadata included in the panoramic image data.

2. The method of claim 1 , wherein the panoramic image data comprises spherical panoramic image data that correspond to a spherical projection plane centered at a location in the building, and wherein each pixel in the spherical panoramic image data includes at least a red component, a green component, a blue component, and a depth component.

3. The method of claim 2 , wherein each pixel in the spherical panoramic image data corresponds to a point on the spherical projection plane associated with a geodesic lattice circumscribed within the spherical projection plane.

4. The method of claim 1 , wherein the metadata comprises a link to a data structure, external to the panoramic image data, that stores performance related data for the building.

5. The method of claim 1 , wherein generating the digital image comprises, for each pixel in the digital image, projecting a ray through a point in the two-dimensional projection plane to select one or more pixels in the panoramic image data that correspond to the ray, and determining a color for the pixel based on the color information included in the one or more pixels.

6. The method of claim 5 , wherein modifying the digital image comprises, for each pixel in the digital image, rendering the pixel with a different color than the color determined for the pixel based on the metadata included in one or more pixels in the panoramic image data.

7. The method of claim 1 , wherein modifying the digital image comprises:
generating a corresponding computer-generated panoramic image data by rendering a graphical representation of the performance related data in a coordinate system corresponding to the panoramic image data; and
for each pixel in the digital image:
determining whether a first depth associated with a corresponding pixel in the panoramic image data is greater than a second depth associated with a corresponding pixel in the computer-generated panoramic image data, and
if the first depth is greater than the second depth, then modifying at least one component of the pixel based on the corresponding pixel in the computer-generated panoramic image data, or
if the first depth is not greater than the second depth, then not modifying the pixel.

8. The method of claim 7 , wherein rendering the graphical representation of the performance related data comprises:
generating a three-dimensional model of one or more components in a CAD (computer-aided drafting) program, wherein the one or more components are defined in the three-dimensional model according to a Cartesian coordinate system; and
generating the computer-generated panoramic image by projecting a plurality of rays from a viewpoint in the Cartesian coordinate system in directions corresponding to the orientation of pixels in the first plurality of pixels relative to the viewpoint, wherein a location of the viewpoint in the three-dimensional model corresponds to a location associated with the panoramic image data.

9. The method of claim 7 , wherein the computer-generated panoramic image data represents a temperature field for at least a portion of the building.

10. A non-transitory computer-readable storage medium containing a program which, when executed by a processor, performs an operation for augmenting panoramic images with performance related data for a building, the operation comprising:
receiving panoramic image data for a spherical panoramic image, the panoramic image data comprising color information and metadata for each pixel in the spherical panoramic image;
generating a digital image comprising a two-dimensional (2D) projection plane based on at least a portion of the spherical panoramic image; and
modifying the digital image to include a graphical representation of performance related data based on the metadata included in the panoramic image data.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the panoramic image data comprises spherical panoramic image data that correspond to a spherical projection plane centered at a location in the building, and wherein each pixel in the spherical panoramic image data includes at least a red component, a green component, a blue component, and a depth component.

12. The non-transitory computer-readable storage medium of claim 11 , wherein each pixel in the spherical panoramic image data corresponds to a point on the spherical projection plane associated with a geodesic lattice circumscribed within the spherical projection plane.

13. The non-transitory computer-readable storage medium of claim 10 , wherein the metadata comprises a link to a data structure, external to the panoramic image data, that stores performance related data for the building.

14. The non-transitory computer-readable storage medium of claim 10 , wherein generating the digital image comprises, for each pixel in the digital image, projecting a ray through a point in the two-dimensional projection plane to select one or more pixels in the panoramic image data that correspond to the ray, and determining a color for the pixel based on the color information included in the one or more pixels.

15. The non-transitory computer-readable storage medium of claim 14 , wherein modifying the digital image comprises, for each pixel in the digital image, rendering the pixel with a different color than the color determined for the pixel based on the metadata included in one or more pixels in the panoramic image data.

16. The non-transitory computer-readable storage medium of claim 10 , wherein modifying the digital image comprises:
generating a corresponding computer-generated panoramic image data by rendering a graphical representation of the performance related data in a coordinate system corresponding to the panoramic image data; and
for each pixel in the digital image:
determining whether a first depth associated with a corresponding pixel in the panoramic image data is greater than a second depth associated with a corresponding pixel in the computer-generated panoramic image data, and
if the first depth is greater than the second depth, then modifying at least one component of the pixel based on the corresponding pixel in the computer-generated panoramic image data, or
if the first depth is not greater than the second depth, then not modifying the pixel.

17. The non-transitory computer-readable storage medium of claim 16 , wherein rendering the graphical representation of the performance related data comprises:
generating a three-dimensional model of one or more components in a CAD (computer-aided drafting) program, wherein the one or more components are defined in the three-dimensional model according to a Cartesian coordinate system; and
generating the computer-generated panoramic image by projecting a plurality of rays from a viewpoint in the Cartesian coordinate system in directions corresponding to the orientation of pixels in the first plurality of pixels relative to the viewpoint, wherein a location of the viewpoint in the thr

Assignees

• Sahoo Keshaba Chandra
• Attar Ramtin
• Poelman Ronald
• Khan Azam
• Autodesk Inc

Inventors

• Sahoo Keshaba Chandra
• Attar Ramtin
• Poelman Ronald
• Khan Azam

Classifications

• G09G2340/12

  Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels · CPC title

• G09G2340/10

  Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels · CPC title

• G09G5/026

  Control of mixing and/or overlay of colours in general (G09G5/022 and G09G5/024 take precedence) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title