Systems and methods for coherent monitoring

Patent metadata

Field	Value
Publication number	US-11727317-B2
Application number	US-202017102215-A
Country	US
Kind code	B2
Filing date	Nov 23, 2020
Priority date	Jan 31, 2019
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Systems and methods are provided for intelligently monitoring environments, classifying objects within such environments, detecting events within such environments, receiving and propagating input concerning image information from multiple users in a collaborative environment, identifying and responding to situational abnormalities or situations of interest based on such detections and/or user inputs.

First claim

The invention claimed is:

1. A system for intelligently monitoring an environment, comprising: one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to:
obtain content representing an environment, the content comprising a plurality of frames, wherein the content comprises video content including the object captured from two angles;
identify, based on the content, one or more discrete objects observed within the environment;
track an object of the one or more discrete objects across the frames;
detect one or more events associated with the tracked object;
learn one or more patterns associated with the detected one or more events;
determine whether a second event deviates from the learned one or more patterns;
in response to determining that the second event deviates from the learned one or more patterns, flag the second event;
generate a three-dimensional (3D) model from the two angles of the object according to a 3D reconstruction algorithm; and
augment a map of the environment with the generated 3D model.

2. The system of claim 1 , wherein the learning the one or more patterns comprises learning seasonal changes in the one or more patterns.

3. The system of claim 1 , wherein the learning the one or more patterns associated with the detected one or more events comprises learning a range associated with the detected one or more events; and the determining whether a second event deviates from the learned one or more patterns comprises determining whether the second event is outside the range.

4. The system of claim 1 , wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
present the map of the environment; and
simultaneously present a playback of the tracked object in a separate pane.

5. The system of claim 4 , wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
identify a geographic boundary associated with the playback of the tracked object;
determine a corresponding boundary in the map;
detect an input to zoom in the geographic boundary; and
change the corresponding boundary in the map based on the input to zoom in.

6. The system of claim 1 , wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
detect a number of occupants within the object based on a heat signature from thermal imagery; and
detect one or more attributes about an occupant of the occupants.

7. The system of claim 1 , wherein the detecting one or more events associated with the tracked object comprises:
detecting changes in the environment;
identifying candidate events based on the detected changes; and
comparing the candidate events with templates while accounting for a scaling, angle, or orientation difference between the object and corresponding objects in the templates.

8. The system of claim 1 , wherein the instructions further cause the system to:
determine a view field of a sensor capturing the content.

9. The system of claim 1 , wherein the instructions further cause the system to:
in response to detecting the one or more events, present a snapshot of a particular frame corresponding to the one or more detected events.

10. A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:
obtaining content representing an environment, the content comprising a plurality of frames, wherein the content comprises video content including the object captured from two angles;
identifying, based on the content, one or more discrete objects observed within the environment;
tracking an object of the one or more discrete objects across the frames;
detecting one or more events associated with the tracked object;
learning one or more patterns associated with the detected one or more events;
determining whether a second event deviates from the learned one or more patterns;
in response to determining that the second event deviates from the learned one or more patterns, flagging the second event;
generate a three-dimensional (3D) model from the two angles of the object according to a 3D reconstruction algorithm; and
augment a map display of the environment with the generated 3D model.

11. The method of claim 10 , wherein the learning the one or more patterns comprises learning seasonal changes in the one or more patterns.

12. The method of claim 10 , wherein the learning the one or more patterns associated with the detected one or more events comprises learning a range associated with the detected one or more events; and the determining whether a second event deviates from the learned one or more patterns comprises determining whether the second event is outside the range.

13. The method of claim 10 , further comprising:
presenting the map of the environment; and
simultaneously presenting a playback of the tracked object in a separate pane.

14. The method of claim 13 , further comprising:
identifying a geographic boundary associated with the playback of the tracked object;
determining a corresponding boundary in the map;
detecting an input to zoom in the geographic boundary; and
changing the corresponding boundary in the map based on the input to zoom in.

15. The method of claim 10 , further comprising:
detecting a number of occupants within the object based on a heat signature from thermal imagery; and
detecting one or more attributes about an occupant of the occupants.

16. The method of claim 10 , wherein the detecting one or more events associated with the tracked object comprises:
detecting changes in the environment;
identifying candidate events based on the detected changes; and
comparing the candidate events with templates while accounting for a scaling, angle, or orientation difference between the object and corresponding objects in the templates.

17. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
obtaining content representing an environment, the content comprising a plurality of frames, wherein the content comprises video content including the object captured from two angles;
identifying, based on the content, one or more discrete objects observed within the environment;
tracking an object of the one or more discrete objects across the frames;
detecting one or more events associated with the tracked object;
learning one or more patterns associated with the detected one or more events;
determining whether a second event deviates from the learned one or more patterns; and
in response to determining that the second event deviates from the learned one or more patterns, flagging the second event;
generating a three-dimensional (3D) model from the two angles of the object according to a 3D reconstruction algorithm; and
augmenting the map of the environment with the generated 3D model.

Assignees

• Palantir Technologies Inc

Inventors

• Cervelli Daniel
• Gupta Anand
• Elder Andrew
• Imig Robert
• Ramalingam Praveen Kumar
• Glidden Reese
• Fedderly Matthew

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06Q10/00 Primary

  Administration; Management · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title