System and method for forming a video stream containing GIS data in real-time

Patent metadata

Field	Value
Publication number	US-10325350-B2
Application number	US-201213492107-A
Country	US
Kind code	B2
Filing date	Jun 8, 2012
Priority date	Jun 10, 2011
Publication date	Jun 18, 2019
Grant date	Jun 18, 2019

Abstract

Image capture systems and methods may include one or more video capture devices capable of capturing one or more video frames. The video frames may include geographic position data and orientation data associated therewith, and may be stored on one or more non-transient computer readable medium. The computer system may marshal each video frame to one or more processors from a bank of processors for geo-referencing and overlaying of geographic information system (GIS) data on the video frames in real time.

First claim

What is claimed is:

1. A method, comprising the steps of:
capturing one or more video frames of a geographic area with one or more video capture devices, the video frames having pixels;
storing the video frames on one or more non-transient computer readable medium accessible by a computer system;
associating geographic position data and orientation data of the one or more video capture devices with each video frame;
marshaling, by the computer system, the video frames to one or more processors from a bank of processors of the computer system;
analyzing the geographic position data and orientation data and the video frames to generate geo-referencing data for pixels of the video frames with the bank of processors, wherein the geo-referencing data comprises geographic locations of pixels of the video frame;
determining a geographical boundary of the video frame from the geo-referencing data;
obtaining one or more layers of geographic information system (GIS) data using the determined geographical boundary of the video frame; and
determining with the bank of processors overlay position of the geographic information system (GIS) data on the video frames in real time based at least in part on the geo-referencing data.

2. The method of claim 1 , wherein the one or more video frames includes one or more series of video frames.

3. The method of claim 1 , wherein the one or more video frames is captured at a full-motion video frame rate.

4. The method of claim 1 , wherein one or more of the video capture devices are secured to a moving platform.

5. The method of claim 4 , wherein one or more of the video capture devices are mounted on a stationary platform.

6. The method of claim 1 , wherein storing the video frames further includes storing interior geometry calibration data for the one or more video capture devices.

7. The method of claim 6 , wherein geo-reference data for the video frames includes utilizing one or more of the geographic position data, the orientation data, and the interior geometry calibration data.

8. The method of claim 1 , further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data, and storing the composite video frames in real-time.

9. The method of claim 8 , further comprising the steps of assembling the video frames into a full motion video series and enabling display of the full motion video series in real-time.

10. The method of claim 8 , wherein the composite video frames are stored over the video frames.

11. The method of claim 8 , wherein the composite video frames are stored separately from the video frames.

12. The method of claim 1 , further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data and displaying one or more composite video frames in non-real-time and one or more composite video frames in real-time.

13. The method of claim 1 , further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data and providing the composite video frames via a website in real-time.

14. The method of claim 1 , wherein the overlay position includes one or more video frame pixel row and one or more video frame pixel column.

15. A method, comprising the steps of:
capturing a series of video frames of a geographic area with one or more cameras from a moving platform and storing the video frames on one or more non-transitory computer readable medium accessible by a computer system while also recording geographic position data and orientation data of the one or more camera on the one or more non-transitory computer readable medium, and with interior geometry calibration data for the one or more cameras being stored on the one or more non-transitory computer readable medium, the video frames having pixels;
performing in real-time the following steps by the computer system for the video frames:
associating the geographic position data and orientation data with the video frames;
analyzing pixels of the video frames to generate geo-referencing data for one or more video frames utilizing the geographic position data, the orientation data and the interior geometry calibration data, wherein the geo-referencing data comprises geographic locations of pixels of the video frame;
determining geographical boundaries of the video frames from the geo-referencing data;
accessing geographic information system (GIS) data using the determined geographical boundary of the video frames; and
determining an overlay position of the GIS data relative to the video frames for which geo-referencing data is generated in real-time based at least in part on the generated geo-reference data.

16. The method of claim 15 , further comprising the steps of overlaying the GIS data on the video frames utilizing the overlay position of the GIS data to form composite video frames having overlaid GIS data and storing the composite video frames in real-time.

17. The method of claim 16 , further comprising the steps of assembling the video frames into a full motion video series and enabling display of the full motion video series in real-time.

18. The method of claim 16 , wherein the composite video frames are stored over the video frames.

19. The method of claim 16 , wherein the composite video frames are stored separately from the video frames.

20. The method of claim 15 , wherein the step of generating geo-referencing data includes marshaling video frames to multiple processors in a predetermined sequence.

21. The method of claim 20 , wherein the predetermined sequence is a round-robin sequence.

22. The method of claim 15 , wherein the step of geo-referencing data includes the step of calculating an offset of a first video frame relative to a second video frame, and using the offset to generate the geo-referencing data for at least one of the first and second video frames.

23. An image capture system comprising:
one or more cameras adapted to be mounted to a moving platform, the one or more cameras adapted to capture a series of video frames of a geographic area, the video frames having pixels;
one or more position and orientation system adapted to record geographic position data and orientation data; and
a computer system having one or more processors and one or more non-transitory memory storing processor executable code and interior geometry calibration data for the one or more cameras, and communicating with the one or more processor;
wherein the processor executable code, when executed by the one or more processors causes the one or more processors to (1) receive a series of video frames from the one or more cameras, (2) store the video frames in the one or more non-transitory memory, (3) record geographic position data and orientation data of the one or more camera on the one or more non-transitory memory, (4) associate the geographic position data and orientation data with the video frames, (5) analyze pixels of the video frames to generate geo-referencing data for each video frame utilizing the geographic position data, the orientation data, and the interior geometry calibration data, wherein the geo-referencing data comprises geographic locations of pixels of the video frame; (6) determine a geographical boundary of the video

Assignees

• Schultz Stephen
• Pictometry Int Corp

Inventors

• Schultz Stephen

Classifications

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T1/0007

  Image acquisition · CPC title