System and method for enhanced surveillance using video analytics

Patent metadata

Field	Value
Publication number	US-11640723-B2
Application number	US-202117149180-A
Country	US
Kind code	B2
Filing date	Jan 14, 2021
Priority date	Oct 20, 2020
Publication date	May 2, 2023
Grant date	May 2, 2023

Abstract

A method of monitoring an aircraft interior includes capturing an image using at least one camera mounted within the fuselage of the aircraft, the image being a captured image including individual image frames. The method further includes modifying the captured image and generating an optimized image using an image processing module, detecting an animate object within the optimized image and identifying features of the animate object using an object detection module, and displaying the optimized using a display module.

First claim

The invention claimed is:

1. A method of monitoring an aircraft interior, the method comprising, in order:
determining a frame extraction pattern;
capturing a video signal from at least one camera mounted within a fuselage of the aircraft, the video signal comprising individual image frames;
modifying the video signal and generating an optimized image using an image processing module, wherein modifying the video signal comprises extracting, based on the frame extraction pattern, select individual frames of the video signal using a filter module;
detecting a human within the optimized image and identifying features of the human using an object detection module, wherein identifying features of the human comprises:
detecting and analyzing a face of the human using a facial detection module; and
detecting facial expressions of the face indicative of a mood of the human using a pre-trained machine learning algorithm of the facial detection module;

analyzing features or motion data of the human using an activity classifier module; and
displaying the optimized image using a display module.

2. The method of claim 1 and further comprising: generating an alert signal using an alert module if the activity classifier module identifies an alertable activity based at least partially on the identified features or motion data.

3. The method of claim 2 , wherein generating an alert signal comprises at least one of activating an indicator light and positioning a bounding box on a portion of the optimized image displayed by the display module.

4. The method of claim 2 and further comprising: at least partially basing identification of the alertable activity on a flight phase of the aircraft.

5. The method of claim 1 , wherein modifying the video signal further comprises at least one of:
enhancing the video signal using an image enhancement module;
performing morphological operations on the video signal using a morphological processing module;
partitioning the video signal using an image segmentation module; and
distinguishing a foreground of the video signal from a background of the image using an extraction module.

6. The method of claim 1 , wherein the object detection module is further configured to detect an inanimate object such as an aircraft fixture.

7. The method of claim 6 , wherein the human is an aircraft passenger.

8. The method of claim 7 , wherein detecting and analyzing the human further comprises at least one of:
detecting a head of the passenger and motion of the head using a head detection module;
detecting a body of the passenger and motion of the body using a body detection module;
detecting a hand of the passenger and motion of the hand using a hand detection module; and
detecting a proximity of the hand to the inanimate object.

9. The method of claim 1 and further comprising: comparing image data to pre-programmed data within an object database.

10. The method of claim 1 , wherein the frame extraction pattern is:
based on frames per unit of time; or
based on a random selection.

11. An aircraft surveillance system comprising:
at least one camera mounted within a fuselage of the aircraft and configured to capture a video signal comprising individual image frames;
an image processing module configured to modify the video signal to generate an optimized image, the image processing module comprising an image filter module configured to extract select individual frames of the video signal based on a frame extraction pattern determined prior to capturing the video signal;
an object detection module configured to detect a human within the optimized image to identify features of the human, wherein identifying features of the human comprises:
detecting and analyzing a face of the human using a facial detection module; and
detecting facial expressions of the face indicative of a mood of the human using a pre-trained machine learning algorithm of the facial detection module;

an activity classifier module configured to analyze features and motion data of the human; and
a display module in data communication with the activity classifier and configured to display the optimized image.

12. The system of claim 11 and further comprising: an alert module configured to generate an alert signal if the activity classifier module identifies an alertable activity based at least partially on the identified features or motion data.

13. The system of claim 12 , wherein the alert signal comprises at least one of an indicator light and a bounding box on a portion of the optimized image displayed on the display device.

14. The system of claim 11 , wherein the image processing module further comprises at least one of:
an image enhancement module;
a morphological processing module;
and image segmentation module; and
an extraction module configured to distinguish an image foreground from an image background.

15. The system of claim 11 and further comprising:
an object database; and
a flight phase indicator module configured to provide information about a flight phase of the aircraft.

16. The system of claim 11 , wherein the human is an aircraft passenger, and wherein the object detection module further comprises at least one of:
a head detection module configured to detect a head of the passenger and motion of the head;
a body detection module configured to detect a body of the passenger and motion of the body; and
a hand detection module configured to detect a hand of the passenger and motion of the hand.

17. The system of claim 16 , wherein the hand detection module is further configured to detect a proximity of the hand to an inanimate object such as a door.

18. The system of claim 11 and further comprising: an un-clustered learning module configured to analyze object and motion data to identify new activities.

19. The system of claim 11 , wherein the at least one camera comprises a plurality of cameras.

20. The system of claim 11 , wherein the frame extraction pattern is:
based on frames per unit of time; or
based on a random selection.

Assignees

• Rosemount Aerospace Inc

Inventors

• Goyal Nitin Kumar
• Pesik Joseph T

Classifications

• G06V40/161 Primary

  Detection; Localisation; Normalisation · CPC title

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• H04N7/183

  for receiving images from a single remote source · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06V20/59

  inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions · CPC title