Systems and methods for generating time discrete 3D scenes

Patent metadata

Field	Value
Publication number	US-10593104-B2
Application number	US-201615394689-A
Country	US
Kind code	B2
Filing date	Dec 29, 2016
Priority date	Dec 31, 2015
Publication date	Mar 17, 2020
Grant date	Mar 17, 2020

Abstract

Methods, systems, and apparatus including medium-encoded computer program products for generating and visualizing 3D scenes include, in one aspect, a method including: obtaining site data acquired by one or more capture devices, wherein the site data comprises data sets corresponding to two or more locations about a physical site, and each respective data set comprises (i) imaging data of the physical site, (ii) coordinate data for the imaging data, and (iii) time metadata for the imaging data; reconstructing a series of three dimensional (3D) modeled scenes of the physical site from the site data using the imaging data, the coordinate data, and the time metadata; receiving a request having associated position, orientation and time data; and generating, in response to the request, output for display of a portion of the 3D modeled scenes to represent the physical site based on the position, orientation and time data.

First claim

What is claimed is:

1. A method comprising:
obtaining site data acquired by one or more capture devices, wherein the site data comprises data sets corresponding to two or more locations about a physical site, and each respective data set comprises (i) imaging data of the physical site, (ii) coordinate data for the imaging data, and (iii) time metadata for the imaging data;
reconstructing a series of three dimensional (3D) modeled scenes of the physical site from the site data using the imaging data, the coordinate data, and the time metadata, wherein the reconstructing comprises, for each one of the 3D modelled scenes in the series, generating a respective 3D model, including textures for the respective 3D model, from source images that represent differing visual perspectives of the physical site;
receiving a request having associated position, orientation and time data, wherein the time data indicates a time period; and
generating, in response to the request, output for display of a portion of the 3D modeled scenes to represent the physical site based on the position, orientation and time data associated with the request, wherein the portion of the 3D modeled scenes is determined by the time period indicated by the request.

2. The method of claim 1 , wherein the one or more capture devices comprise multiple capture devices including at least one camera that captures two dimensional (2D) images as the imaging data, and at least one scanner that captures 3D range data as the imaging data, wherein each of the multiple capture devices provides a different perspective of the physical site, and wherein generating the respective 3D model for at least one of the 3D modelled scenes in the series comprises:
extracting a dense 3D point cloud from the 2D images;
computing a coarse mesh from the dense 3D point cloud;
refining the coarse mesh on the 2D images to form an updated mesh of the at least one of the 3D modelled scenes; and
computing a UV atlas of the updated mesh for use in texture creation for the at least one of the 3D modelled scenes in the series.

3. The method of claim 1 , wherein the one or more capture devices comprise a single capture device being an unmanned aerial vehicle that is positioned at the two or more locations about the physical site to provide different perspectives of the physical site.

4. The method of claim 1 , wherein the one or more capture devices are programed to acquire the site data at predetermined capture times, wherein the predetermined capture times are separated by an hourly, daily, weekly or bi-monthly time interval within a selected time period that is at least one month, and wherein the physical site is a building construction site.

5. The method of claim 4 , wherein the request is received from a mobile device located at the physical site, wherein the output for display of the portion of the 3D modeled scenes is transmitted to the mobile device for use in displaying the portion of the 3D modeled scenes at the mobile device, and wherein the portion of the 3D modeled scenes is at least one time slice from the predetermined capture times within the selected time period that falls within the time period indicated by the request.

6. The method of claim 5 , comprising creating for display on a user interface element of the mobile device an animation of movement in 3D space and time, based on user specified paths in 3D space and time transitions, with respect to the portion of the 3D modeled scenes displayed at the mobile device, where textures are blended together to transition between time periods in the animation, and the movement in 3D space and time is across time-sliced 3D visualizations of the physical site to show changes of the physical site over time.

7. The method of claim 5 , comprising iteratively generating two or more perspective discrete outputs for display of the portion of the 3D modeled scenes based on additionally received position and orientation data, wherein the additionally received position and orientation data is generated through motion input via physical movement of the mobile device.

8. The method of claim 5 , comprising iteratively generating two or more time discrete outputs for display, wherein the two or more time discrete outputs correspond to portions of the 3D modeled scenes with time metadata associated with additionally received time data through a user interface element of the mobile device, the time metadata comprises timestamps, and the portions of the 3D modeled scenes is two or more time discrete 3D scenes representing selected days during a lifecycle of a construction project.

9. The method of claim 8 , wherein the two or more time discrete outputs are displayed as two dimensional (2D) images or 3D scenes at the mobile device.

10. The method of claim 8 , wherein the iteratively generating the two or more time discrete outputs for display comprises generating a modeled scene to be superimposed, using semitransparency, onto live imaging data of the physical site currently captured by the mobile device, wherein the superimposed modeled scene is displayed as 2D images or 3D scenes at the mobile device.

11. The method of claim 1 , wherein the physical site is a first physical site of a first project, the series of 3D modeled scenes are a series of first 3D modeled scenes in a first coordinate space, and the method comprising:
loading a series of second 3D modeled scenes in a second coordinate space for a second physical site of a second project, the second physical site of the second project being a different site from the first physical site of the first project; and
registering the first coordinate space with the second coordinate space;
wherein the generating comprises superimposing, using semitransparency, the portion of the first 3D modeled scenes of the first physical site on a portion of the second 3D modeled scenes of the second physical site.

12. A system comprising:
a computer network;
one or more capture devices communicatively coupled with the computer network, wherein the one or more capture devices are positioned about a physical site and programed to acquire site data associated with two or more locations about the physical site;
one or more computing devices communicatively coupled with the computer network and programed to (i) obtain the site data acquired by the one or more capture devices, wherein the site data comprises data sets corresponding to the two or more locations about the physical site, and each respective data set comprises (a) imaging data of the physical site, (b) coordinate data for the imaging data, and (c) time metadata for the imaging data, (ii) reconstruct a series of three dimensional (3D) modeled scenes of the physical site from the site data using the imaging data, the coordinate data, and the time metadata, wherein the one or more computing devices are programed to, for each one of the 3D modelled scenes reconstructed in the series, generate a respective 3D model, including textures for the respective 3D model, from source images that represent differing visual perspectives of the physical site, (iii) receive a request having associated position, orientation and time data, wherein the time data indicates a time period; and (iv) generate, in response to the request, output for display of a portion of the 3D modeled scenes to represent the physical site based on the position, orientation and time data associated with the request, wherein the portion of the 3D modeled scenes is determined by the time period indicated by the request.

13. The system of claim 12 , wherein the one or more capture devices comprise multiple capture devices including at least

Assignees

• Autodesk Inc

Inventors

• Robert Luc
• Gallo Emmanuel

Classifications

• H04N1/00

  Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T7/55

  from multiple images · CPC title

• G01B11/24 Primary

  for measuring contours or curvatures · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title