Photogrammetry

Patent metadata

Field	Value
Publication number	US-12136163-B2
Application number	US-202117919773-A
Country	US
Kind code	B2
Filing date	May 18, 2021
Priority date	May 19, 2020
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

Measures, including a method, system, and computer program, for performing in-mission photogrammetry while an imaging platform is surveying a target environment. A plurality of images of the target environment are captured by the imaging platform. At the imaging platform, a representative subset of images is identified from the captured plurality of images. The imaging platform transmits the identified representative subset to a processing node. At the processing node, the transmitted representative subset is processed to generate a virtual model of the target environment.

First claim

The invention claimed is:

1. A method of performing in-mission photogrammetry, the method comprising, in-mission while an imaging platform is surveying a target environment:
capturing, by the imaging platform, a plurality of images of the target environment;
at the imaging platform, identifying a representative subset of images from the captured plurality of images;
transmitting, by the imaging platform, the identified representative subset to a processing node;
at the processing node, processing the transmitted representative subset to generate a virtual model of the target environment;
receiving, at the processing node, a request for refinement of the virtual model; and
in response to receipt of the request, transmitting an instruction to the imaging platform to capture additional imagery to fulfil the request.

2. A method according to claim 1 , wherein the transmitting comprises streaming the identified representative subset over a communications link, wherein a bandwidth of the communications link precludes streaming of the full captured plurality of images.

3. A method according to claim 1 , wherein the request comprises a request for an increase in a resolution of the virtual model.

4. A method according to claim 1 , wherein the request comprises a request for an extension of the virtual model to include an adjacent further target environment.

5. A method according to claim 1 , wherein the request is transmitted in response to receipt of user input indicating a desired refinement of the virtual model.

6. A method according to claim 5 , further comprising:
importing the generated virtual model into a virtual environment; and
presenting the virtual environment to an end user to enable the end user to interact with the virtual environment, wherein the user input is provided within the virtual environment by the end user.

7. A method according to claim 1 , further comprising:
importing the generated virtual model into a virtual environment; and
presenting the virtual environment to an end user to enable the end user to interact with the virtual environment.

8. A method according to claim 7 , wherein the importing comprises processing the virtual model to remove lighting effects resulting from lighting conditions during the capturing.

9. A method according to claim 8 , wherein the importing comprises applying one or more desired lighting and/or weather effects to the virtual model.

10. A method according to claim 7 , wherein the importing comprises processing the virtual model to generate a collision mesh, the collision mesh defining one or more surfaces of the virtual environment for collision detection.

11. A method according to claim 7 , wherein the importing comprises:
analyzing the generated virtual model to identify one or more objects of interest in the virtual model; and
in response to the analyzing indicating an object of interest, tagging the object in the virtual environment.

12. A method according to claim 1 , wherein the identifying comprises running an optimization algorithm to reduce information loss.

13. A method according to claim 1 , wherein the identifying is performed at least partially contemporaneously with the capturing.

14. A method according to claim 1 , wherein the processing comprises:
processing the transmitted representative subset of images to produce a three dimensional point cloud associated with a geometry of the target environment;
based on the produced point cloud, generating a mesh representing a geometry of the target environment; and
applying the representative subset to the generated mesh to generate a textured mesh representing the target environment.

15. A method according to claim 1 , wherein:
the representative subset of images comprises respective timestamps;
the generation is performed based on the timestamps, such that the virtual model represents the target environment at multiple distinct points in time.

16. A method according to claim 15 , comprising:
receiving time indication user input from an end user, the time indication user input indicating a desired one of the points in time; and
in response to the receipt of the time indication user input, presenting the virtual model such that it represents the indicated point in time.

17. A method according to claim 16 , further comprising:
importing the generated virtual model into a virtual environment;
presenting the virtual environment to an end user to enable the end user to interact with the virtual environment; and
updating the virtual environment to use the virtual model associated with the indicated point in time.

18. A system for performing in-mission photogrammetry comprising:
an imaging platform configured to capture a plurality of images of a target environment, identify a representative subset of images from the captured plurality of images, and transmit the identified representative subset to a processing node; and
a processing node configured to process, in-mission while the imaging platform is surveying the target environment, the transmitted representative subset to generate a virtual model of the target environment, to receive a request for refinement of the virtual model, and, in response to receipt of the request, to transmits an instruction to the imaging platform to capture additional imagery to fulfil the request.

19. A non-transitory computer readable medium comprising a set of instructions, which, when executed by one or more computerized devices, cause the computerized devices to perform a method of performing in-mission photogrammetry, the method comprising, in-mission while an imaging platform is surveying a target environment:
capturing, by the imaging platform, a plurality of images of the target environment;
at the imaging platform, identifying a representative subset of images from the captured plurality of images;
transmitting, by the imaging platform, the identified representative subset to a processing node;
at the processing node, processing the transmitted representative subset to generate a virtual model of the target environment;
receiving, at the processing node, a request for refinement of the virtual model; and
in response to receipt of the request, transmitting an instruction to the imaging platform to capture additional imagery to fulfil the request.

Assignees

• Airbus Defence & Space Ltd

Inventors

• Taylor Stuart

Classifications

• G06T15/50

  Lighting effects · CPC title

• G06T2207/30232

  Surveillance · CPC title

• G06T2207/30212

  Military · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T7/97

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title