Method for detecting and classifying events of a scene that include firing threat type events

Patent metadata

Field	Value
Publication number	US-9953226-B2
Application number	US-201515110937-A
Country	US
Kind code	B2
Filing date	Feb 5, 2015
Priority date	Feb 7, 2014
Publication date	Apr 24, 2018
Grant date	Apr 24, 2018

Abstract

The subject of the invention is a method for detecting and classifying events of a scene by means of a single-pupil imaging system equipped with a VisNIR detector in the 0.6 μm-1.1 μm band and with an SWIR detector, which comprises steps of acquiring synchronized VisNIR and SWIR successive 2D images, of displaying the VisNIR images, and of processing these images, which consists in: comparing the SWIR images so as to determine, for each pixel, the variation in illumination from one SWIR image to another and the peak value of these SWIR illuminations, if this variation in SWIR illumination is greater than a threshold, then an event associated with said pixel is detected and: its date, its temporal shape and its duration are determined, in the VisNIR images, the coordinates are determined of the corresponding pixel for which: the variation in the illumination from one VisNIR image to another and the peak value of these VisNIR illuminations are calculated, and these variations in SWIR and VisNIR illumination and their peak values are compared so as to estimate a temperature of the event, the distance of the corresponding point of the scene is estimated so as to calculate the intensity of the event on the basis of the SWIR and VisNIR illuminations and on the basis of this distance, the total energy of the event is estimated on the basis of its temporal shape and of its intensity, the event is classified as a function of its duration, its temperature, its intensity and its energy, the previous steps are repeated for another pixel of the SWIR images.

First claim

The invention claimed is:

1. A method for detecting and classifying events of firing threats type of a scene, the method comprising:
providing a single-pupil imaging system mounted on a mobile platform and equipped with several detectors, including a detector configured to operate in the 0.6 μm-1.1 μm wavelength band that comprises a Visible Near Infra Red (VisNIR) detector and a detector configured to operate in the 0.9 μm-1.7 μm wavelength band that comprises a Short Wave Infra Red (SWIR) detector, associated with a detection processing unit, which is configured to implement a step of acquiring successive 2D images of the scene which arise from the VisNIR detector that comprise VisNIR images, and successive 2D images of the scene which arise from the SWIR detector and are synchronized with the VisNIR images that comprise SWIR images, a step of displaying the VisNIR images, and a step of processing the VisNIR and SWIR images via the detection processing unit, wherein the step of processing the images comprises the following sub-steps:
comparing the successive SWIR images so as to determine, for each pixel (x 1 , y 1 ) and neighboring pixels, a variation in illumination from one SWIR image to another SWIR image and determine a peak value e 1 (t) of SWIR illuminations,
if the variation in SWIR illumination is greater than a predetermined illumination threshold, then the variation in SWIR illumination during which said threshold is exceeded is designated as a signature of an event i and an event is associated with said pixel (x 1 , y 1 ) or with a barycenter of the pixels considered, and the step of processing the images further comprises the sub-steps of:
determining a date t i of the event i,
determining a temporal shape and a duration of the signature of the event,
determining the coordinates of a pixel (x 2 , y 2 ) and of neighboring pixels corresponding to the pixel (x 1 , y 1 ) or to a barycenter in the VisNIR images synchronized with the SWIR images, and for the pixel (x 2 , y 2 ) and the neighboring pixels:
calculating a variation in the illumination from one VisNIR image to another VisNIR image and calculate a peak value e 2 (t) of VisNIR illuminations, comparing these variations in SWIR and VisNIR illumination, and comparing the peak values e 1 (t) and e 2 (t) so as to estimate a temperature of the event on a basis of a predetermined lookup table,

if the scene is a daytime scene and the temperature is greater than a temperature threshold, then the event is a false alarm and the previous steps from the comparing step are repeated with another pixel of the SWIR images,
otherwise:
estimating a distance Ri of a corresponding point of the real-time scene on a basis of measurements of angular speeds of elements of the scene, of a speed of the platform, and of the VisNIR images,
calculating an intensity I i of the event i on a basis of the SWIR and VisNIR illuminations of the pixel and on a basis of the distance R i ,
estimating a total energy E i of the event on a basis of a temporal shape of the signature and of the intensity I i ,
classifying the event i as a function of its duration δt i , its temperature T i , its intensity I i and its energy E i ,
estimating an effectiveness range Pi of the event i on a basis of its classification,
comparing the effectiveness range Pi with a distance Ri,
if the distance Ri is less than the effectiveness range Pi, then if possible, triggering a suitable retaliation in real time.

2. The method for classifying events of a scene as claimed in claim 1 , wherein when an event is associated with adjacent pixels, then a luminance of a set of these events is determined, and this set of events is also classified as a function of its luminance.

3. The method for classifying events of a scene as claimed in claim 1 , wherein the event is inlayed on the VisNIR image displayed.

4. The method for classifying events of a scene as claimed in claim 3 , wherein the distance associated with the event is displayed on the VisNIR image displayed.

5. The method for classifying events of a scene as claimed in claim 1 , wherein the retaliation is triggered automatically or manually by an operator.

6. A non-transitory computer program product, said non-transitory computer program product comprising code instructions configured to perform the steps of the method as claimed in claim 1 , when said non-transitory computer program is executed on a computer.

7. A system for detecting and classifying events of a scene, which comprises:
a system for single-pupil imaging of the scene, mounted on a mobile platform and equipped with several detectors, including a Visible Near Infra Red (VisNIR) detector and a Short Wave Infra Red (SWIR) detector,
a detection processing unit linked to the detectors,
means for estimating the distance between points of the scene and the imaging system,
an events management system linked to the detection processing unit and configured so as to be triggered, on the basis of the classified event and of its distance,
a display device linked to the detection processing unit,
the detection processing unit being configured to compare the successive SWIR images so as to determine, for each pixel (x 1 , y 1 ) and neighboring pixels, a variation in illumination from one SWIR image to another SWIR image and determine a peak value e 1 (t) of SWIR illuminations,
if the variation in SWIR illumination is greater than a predetermined illumination threshold, then the variation in SWIR illumination during which said threshold is exceeded is designated as a signature of an event i and an event is associated with said pixel (x 1 , y 1 ) or with a barycenter of the pixels considered, and the detection processing unit configured to:
determine a date t i of the event i,
determine a temporal shape and a duration of the signature of the event,
determine the coordinates of a pixel (x 2 , y 2 ) and of neighboring pixels corresponding to the pixel (x 1 , y 1 ) or to a barycenter in the VisNIR images synchronized with the SWIR images, and for the pixel (x 2 , y 2 ) and the neighboring pixels:

the detection processing unit is further configured to calculate a variation in the illumination from one VisNIR image to another VisNIR image and calculate a peak value e 2 (t) of VisNIR illuminations, comparing these variations in SWIR and VisNIR illumination, and comparing the peak values e 1 (t) and e 2 (t) so as to estimate a temperature of the event on a basis of a predetermined lookup table,
if the scene is a daytime scene and the temperature is greater than a temperature threshold, then the event is a false alarm and the detection processing unit is configured to repeat the comparison with another pixel of the SWIR images, otherwise:
the detection processing unit is further configured to estimate a distance Ri of a corresponding point of the real-time scene on a basis of measurements of angular speeds of elements of the scene, of a speed of the platform, and of the VisNIR images,
the detection processing unit is further configured to calculate an intensity I i of the event i on a basis of the SWIR and VisNIR illuminations of the pixel and on a basis of the distance R i ,
the detection processing unit is further configured to estimate a total energy E i of the event on a basis of a temporal shape of the signature and of the intensity I i ,
the detection processing unit is further configured to classify the event i as a function of its duration δt i , its temperature T i , its intensity I i and its energy E i ,
the detection processing unit is further configured to estimate an effectiveness range Pi of the event i on a basis of its classification,
the detection processing unit is further configured to compare the effectiveness range Pi

Assignees

• Thales Sa

Inventors

• Midavaine Thierry
• Garin Olivier

Classifications

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V20/00 Primary

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title

• G06K9/00771 Primary

  Physics · mapped topic

• G01J5/0066

  for hot spots detection · CPC title

• G01J5/025

  Interfacing a pyrometer to an external device or network; User interface · CPC title