Area monitoring system implementing a virtual environment

Patent metadata

Field	Value
Publication number	US-10061486-B2
Application number	US-201414533867-A
Country	US
Kind code	B2
Filing date	Nov 5, 2014
Priority date	Nov 5, 2014
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

An area monitoring system includes a camera to monitor a region of interest and provide real-time video data corresponding to real-time video images of the region of interest. A memory stores data associated with a rendered virtual environment associated with a geographic region that includes the region of interest. An image processor combines the real-time video data and the data associated with the virtual environment to generate image data. A user interface enables a user to navigate the virtual environment via inputs and displays the image data based on a location perspective corresponding to a viewing perspective of the user at a given virtual location within the virtual environment. The image processor superimposes the video images of the region of interest onto the virtual environment in the image data at a location and an orientation associated with the location perspective of the user within the virtual environment.

First claim

What is claimed is:

1. An area monitoring system comprising:
at least one camera configured to monitor a region of interest and to provide real-time video data corresponding to real-time video images of the region of interest;
a memory configured to store data associated with a rendered three-dimensional virtual environment that is associated with a geographic region that includes at least the region of interest;
an image processor configured to combine the real-time video data and the data associated with the rendered three-dimensional virtual environment to generate image data; and
a user interface configured to enable a user to navigate the rendered three-dimensional virtual environment via inputs and to display the image data based on a location perspective corresponding to a viewing perspective of the user at a given virtual location within the rendered three-dimensional virtual environment, the image processor being configured to superimpose the real-time video images of the region of interest onto the rendered three-dimensional virtual environment in the image data at a location and an orientation based on the location perspective of the user within the rendered three-dimensional virtual environment, wherein the user interface is further configured to enable the user to view the image data in a dynamic object-centric view associated with the location perspective of the user being offset from and substantially centered upon a dynamic object that is graphically displayed as a dynamic object icon with respect to the rendered three-dimensional virtual environment via the image processor in response to location information associated with the dynamic object.

2. The system of claim 1 , wherein the user interface is further configured to enable the user to view the image data in one of an overhead view associated with the location perspective of the user being an overhead perspective with respect to the rendered three-dimensional virtual environment, and a camera-perspective view associated with the location perspective of the user being substantially similar to a perspective of a respective one of the at least one camera superimposed on the rendered three-dimensional virtual environment.

3. The system of claim 2 , wherein the user interface is further configured to enable the user to preview the perspective of the respective one of the at least one camera from the overhead view.

4. The system of claim 2 , wherein the user interface is further configured to enable the user to select the camera-perspective view by selecting a camera icon via the user interface, the camera icon corresponding to a three-dimensional physical location of the respective one of the at least one camera, the camera icon being superimposed on the rendered three-dimensional virtual environment via the image processor.

5. The system of claim 4 , wherein the image data comprises a visual indication of a field of view of the perspective of each of the at least one camera in the overhead view, wherein the visual indication of the field of view emanates from the camera icon of each respective one of the at least one camera.

6. The system of claim 1 , wherein the user interface is further configured to enable the user to select the dynamic object-centric view by selecting the dynamic object icon via the user interface, wherein the dynamic object-centric view comprises a perspective angle and offset distance of the location perspective of the user relative to the dynamic object that are each adjustable via the user interface.

7. The system of claim 1 , further comprising a dynamic object interface configured to receive at least one dynamic object input associated with a respective dynamic object, the at least one dynamic object input comprising location information associated with the dynamic object, wherein the image processor is configured to superimpose a dynamic object icon in real-time on the rendered three-dimensional virtual environment at a location corresponding to a location of the dynamic object in the geographic region based on the location information.

8. The system of claim 7 , wherein the image processor is further configured to superimpose a tag associated with the respective dynamic object onto the image data, the tag comprising descriptive information associated with the respective dynamic object based on the at least one dynamic object input.

9. The system of claim 1 , further comprising a video controller configured to receive the real-time video data and to provide the real-time video data to the image processor in response to a request for the real-time video data by the image processor based on the location perspective of the user within the rendered three-dimensional virtual environment.

10. The system of claim 9 , wherein the video controller comprises a variable resolution controller configured to provide the real-time video data at a resolution that varies based on at least one of a distance and a viewing perspective angle of the location perspective of the user with respect to a location of the real-time video images superimposed in the rendered three-dimensional virtual environment relative to the location perspective of the user.

11. A non-transitory computer readable medium comprising instructions that, when executed, are configured to implement a method for monitoring a geographic region, the method comprising:
receiving real-time video data corresponding to real-time video images of a region of interest within the geographic region via a plurality of cameras;
accessing data associated with a rendered three-dimensional virtual environment that is associated with the geographic region that includes at least the region of interest from a memory;
receiving navigation inputs provided via a user interface, the navigation inputs corresponding to virtual movement of a user within the rendered three-dimensional virtual environment and to a location perspective corresponding to a viewing perspective of the user at a given virtual location within the rendered three-dimensional virtual environment,
receiving at least one dynamic object input associated with a respective dynamic object, the at least one dynamic object input comprising location information associated with the dynamic object;
superimposing the real-time video images of the region of interest associated with at least one of the plurality of cameras in the location perspective of the user onto the rendered three-dimensional virtual environment in the image data at a location and an orientation based on the location perspective of the user within the rendered three-dimensional virtual environment;
superimposing a dynamic object icon in real-time on the rendered three-dimensional virtual environment at a location corresponding to a location of the dynamic object in the geographic region based on the location information;
implementing the navigation inputs to facilitate selecting the dynamic object icon via the user interface to enable the user to view the image data in a dynamic object-centric view associated with the location perspective of the user being offset from and substantially centered upon the dynamic object icon;
adjusting at least one of a perspective angle and an offset distance associated with the location perspective of the user relative to the dynamic object icon in the dynamic object-centric view to adjust the orientation of the superimposed real-time video images of the region of interest associated with at least one of the plurality of cameras based on the location perspective of the user.

12. The method of claim 11 , wherein receiving navigation inputs comprises receiving navigation inputs via the user interface that

Assignees

• Kaehler Adrian
• Stine Matt
• Selnick Dan
• Northrop Grumman Systems Corp

Inventors

• Kaehler Adrian
• Stine Matt
• Selnick Dan

Classifications

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06T19/003 Primary

  Navigation within 3D models or images · CPC title

• G06F3/04817

  using icons (graphical or visual programming using iconic symbols G06F8/34) · CPC title

• G06F3/04842 Primary

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06K9/00771

  Physics · mapped topic