Systems and methods for coherent monitoring

Patent metadata

Field	Value
Publication number	US-12229692-B2
Application number	US-202318234255-A
Country	US
Kind code	B2
Filing date	Aug 15, 2023
Priority date	Jan 31, 2019
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Systems and methods are provided for intelligently monitoring environments, classifying objects within such environments, detecting events within such environments, receiving and propagating input concerning image information from multiple users in a collaborative environment, identifying and responding to situational abnormalities or situations of interest based on such detections and/or user inputs.

First claim

The invention claimed is:

1. A system for intelligently monitoring an environment, comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to:
obtain content representing an environment, the content comprising a plurality of frames, wherein the content comprises video content including an object captured from two angles;
identify, based on the content, one or more discrete objects observed within the environment;
track the object of the one or more discrete objects across the frames;
detect an event that deviates from one or more patterns;
in response to determining that the event deviates from the one or more patterns, flag the event;
generate a three-dimensional (3D) model from the two angles of the object according to a 3D reconstruction algorithm; and
augment a map of the environment with the generated 3D model.

2. The system of claim 1 , wherein the instructions further cause the system to:
receive an indication of a missed detection of an instance of the object or a different object of the discrete objects;
receive a label of an object type corresponding to the missed detection; and
learn one or more attributes of the missed detection to recognize any future instances of an object type.

3. The system of claim 1 , wherein the instructions further cause the system to:
learn one or more patterns associated with the detected event, wherein the one or more patterns comprise seasonal changes.

4. The system of claim 3 , wherein the learning the one or more patterns associated with the detected event comprises learning a range associated with the detected event; and the determining that the detected event deviates from the learned one or more patterns comprises determining that the detected event is outside of or fails to satisfy a corresponding range.

5. The system of claim 1 , wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
present the map of the environment; and
simultaneously present a playback of the tracked object in a separate pane.

6. The system of claim 5 , wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
identify a geographic boundary associated with the playback of the tracked object;
determine a corresponding boundary in the map;
detect an input to zoom in the geographic boundary; and
change the corresponding boundary in the map based on the input to zoom in.

7. The system of claim 1 , wherein the memory stored instructions that, when executed by the one or more processors, further causes the system to:
detect a number of occupants within the object based on a heat signature from thermal imagery; and
detect one or more attributes about an occupant of the occupants.

8. The system of claim 1 , wherein the detecting one or more events associated with the tracked object comprises:
detecting changes in the environment;
identifying candidate events based on the detected changes; and
comparing the candidate events with templates while accounting for a scaling, angle, or orientation difference between the object and corresponding objects in the templates.

9. The system of claim 1 , wherein the instructions further cause the system to:
determine a view field of a sensor capturing the content.

10. The system of claim 1 , wherein the instructions further cause the system to:
in response to detecting the one or more events, present a snapshot of a particular frame corresponding to the one or more detected events.

11. A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:
obtaining content representing an environment, the content comprising a plurality of frames, wherein the content comprises video content including an object captured from two angles;
identifying, based on the content, one or more discrete objects observed within the environment;
tracking the object of the one or more discrete objects across the frames;
detecting an event that deviates from one or more patterns;
in response to determining that the event deviates from the one or more patterns, flag the event;
generating a three-dimensional (3D) model from the two angles of the object according to a 3D reconstruction algorithm; and
augmenting a map of the environment with the generated 3D model.

12. The method of claim 11 , further comprising:
receiving an indication of a missed detection of an instance of the object or a different object of the discrete objects;
receiving a label of an object type corresponding to the missed detection; and
learning one or more attributes of the missed detection to recognize any future instances of an object type.

13. The method of claim 11 , further comprising:
learning one or more patterns associated with the detected event, wherein the one or more patterns comprise seasonal changes.

14. The method of claim 13 , wherein the learning the one or more patterns associated with the detected event comprises learning a range associated with the detected event; and the determining that the detected event deviates from the learned one or more patterns comprises determining that the detected event is outside of or fails to satisfy a corresponding range.

15. The method of claim 11 , further comprising:
presenting the map of the environment; and
simultaneously presenting a playback of the tracked object in a separate pane.

16. The method of claim 11 , further comprising:
identifying a geographic boundary associated with the playback of the tracked object;
determining a corresponding boundary in the map;
detecting an input to zoom in the geographic boundary; and
changing the corresponding boundary in the map based on the input to zoom in.

17. The method of claim 11 , further comprising:
detecting a number of occupants within the object based on a heat signature from thermal imagery; and
detecting one or more attributes about an occupant of the occupants.

18. The method of claim 11 , wherein the detecting one or more events associated with the tracked object comprises:
detecting changes in the environment;
identifying candidate events based on the detected changes; and
comparing the candidate events with templates while accounting for a scaling, angle, or orientation difference between the object and corresponding objects in the templates.

19. The method of claim 11 , further comprising:
determining a view field of a sensor capturing the content.

20. The method of claim 11 , further comprising:
in response to detecting the one or more events, presenting a snapshot of a particular frame corresponding to the one or more detected events.

Assignees

• Palantir Technologies Inc

Inventors

• Cervelli Daniel
• Gupta Anand
• Elder Andrew
• Imig Robert
• Ramalingam Praveen
• Glidden Reese
• Fedderly Matthew

Classifications

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title