Monitoring system of a passenger conveyor, a passenger conveyor, and a monitoring method thereof

Patent metadata

Field	Value
Publication number	US-10059569-B2
Application number	US-201715660194-A
Country	US
Kind code	B2
Filing date	Jul 26, 2017
Priority date	Jul 29, 2016
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

A monitoring system and a monitoring method for a passenger conveying device, and the passenger conveying device. The monitoring system comprises an imaging sensor and/or a depth sensing sensor for sensing a monitoring area of the passenger conveying device to acquire data frames; and a processing device for performing data processing on the data frames to monitor whether the monitoring area is abnormal, and configured to comprise: a background acquisition module for acquiring a background model based on data frames sensed when the monitoring area is in a normal state; a foreground detection module for comparing data frames sensed in real time with the background model to obtain a foreground object; a foreground feature extraction module for extracting a corresponding foreground object markup feature from the foreground object; and a state judgment module for judging whether the foreground object belongs to an abnormal population.

First claim

What is claimed is:

1. A monitoring system for a passenger conveying device comprising:
an imaging sensor and/or a depth sensing sensor used for sensing a monitoring area of the passenger conveying device to acquire data frames; and
a processing device used for performing data processing on the data frames to monitor whether the monitoring area is abnormal, wherein the processing device is configured to:
perform a background acquisition process, wherein the background acquisition process includes acquiring a background model based on data frames sensed when the monitoring area is in a normal state;
perform a foreground detection process, wherein the foreground detection process includes comparing data frames sensed in real time with the background model to obtain a foreground object;
perform a foreground feature extraction process, wherein the foreground feature extraction process includes:
defining an abnormal human body model based on human body non-self-learning modeling and/or human body self-learning modeling; and
extracting a corresponding foreground object markup feature from the foreground object; and

perform a state judgment process, wherein the state judgment process includes judging whether the foreground object belongs to an abnormal population at least based on the foreground object markup feature.

2. The monitoring system according to claim 1 , wherein the human body self-learning modeling obtains the abnormal human body model through a deep learning technique, and the deep learning technique comprises a deep neural network, a convolutional neural network, a recursive neural network and a dictionary learning or a bag-of-visual-words model; and/or the human body non-self-learning modeling comprises a bone map model and/or a binary map model.

3. The monitoring system according to claim 1 , wherein the extracted foreground object markup feature comprises one or more of a color, a size, a shape and a speed of a human body in the foreground object, and wherein the state judgment process includes judging whether the foreground object belongs to the abnormal population based on one or more of the color, the size, the shape and the speed of the human body in the foreground object.

4. The monitoring system according to claim 3 , wherein foreground object is judged as belonging to the abnormal population when one or more of the color, the size, the shape and the speed of the human body in the foreground object falls into the abnormal human body model.

5. The monitoring system according to claim 1 , wherein the foreground feature extraction process further includes defining an abnormal object model based on object non-self-learning modeling and/or object self-learning modeling.

6. The monitoring system according to claim 5 , wherein the extracted foreground object markup feature comprises a size and/or a shape of an object in the foreground object, wherein the state judgment process includes judging whether the foreground object belongs to the abnormal population based on the size and/or the shape of the object in the foreground object.

7. The monitoring system according to claim 6 , wherein the foreground object is judged as belonging to the abnormal population when the size and/or the shape of the object in the foreground object falls into the abnormal object model.

8. The monitoring system according to claim 1 , wherein:
the foreground feature extraction process further includes extracting a corresponding foreground object markup feature around the abnormal population from the foreground object when the foreground object belongs to the abnormal population; and
the state judgment process further includes judging whether the foreground object is in an abnormal state at least based on the corresponding foreground object markup feature around the abnormal population.

9. The monitoring system according to claim 1 , wherein the foreground feature extraction process further includes defining a pet model based on pet non-self-learning modeling and/or pet self-learning modeling.

10. The monitoring system according to claim 9 , wherein the extracted foreground object markup feature comprises one or more of a shape, a profile, a size and a color of a pet in the foreground object, wherein the state judgment process includes judging whether the foreground object belongs to the abnormal population based on one or more of the shape, the profile, the size and the color of the pet in the foreground object.

11. The monitoring system according to claim 10 , wherein the foreground object is judged as belonging to the abnormal population when the size and/or the shape of the pet in the foreground object falls into the pet model.

12. The monitoring system according to claim 1 , wherein the processing device is further configured to:
perform a trajectory generation process, wherein the trajectory generation process includes generating a change trajectory of the foreground object markup feature according to the foreground object markup feature respectively extracted from the foreground object corresponding to a plurality of continuous data frames.

13. The monitoring system according to claim 12 , the trajectory is generated by applying a Bayesian estimation technique to a vector of the foreground object markup feature.

14. The monitoring system according to claim 12 , wherein the state judgement process further includes pre-judging whether the foreground object will belong to the abnormal population based on the change trajectory of the foreground object markup feature.

15. The monitoring system according to claim 1 , wherein the foreground object is judged as belonging to the abnormal population when judgment results of at least two continuous data frames are that the foreground object belongs to the abnormal population.

16. The monitoring system according to claim 1 , wherein the imaging sensor and/or the depth sensing sensor senses and acquires data frames within a predetermined period at an interval of a predetermined period for the processing device to perform data processing.

17. The monitoring system according to claim 1 , wherein the imaging sensor and/or the depth sensing sensor comprises a plurality of imaging sensors and/or depth sensing sensors, which are disposed to be capable of covering an entrance and/or an exit and/or an entire length direction of the passenger conveying device.

18. The monitoring system according to claim 1 further comprising an alarm unit, wherein the state judgment process further includes triggering the alarm unit when the foreground object is judged as belonging to the abnormal population.

19. The monitoring system according to claim 1 , wherein the processing device is further configured to trigger outputting of a signal to the passenger conveying device and/or an escalator maintenance center of a building to prevent a safety accident from occurring when the foreground object is judged as belonging to the abnormal population.

20. The monitoring system according to claim 1 , wherein the background acquisition process includes performing processing on the sensed data frames to acquire the background model through a Gaussian mixture model, a codebook model, principle components analysis or robust principle components analysis.

21. The monitoring system according to claim 1 , wherein the foreground detection process includes comparing the data frames sensed in real time with the background model through an inter-frame difference, morphological operators and temporal, spatial or

Assignees

• Otis Elevator Co

Inventors

• Fang Hui
• Jia Zhen
• Chen Yanzhi
• Finn Alan Matthew
• Hsu Arthur
• Zhao Jianwei
• Hu Zhaoxia
• Li Qiang
• Su Anna
• Wang Longwen
• Guo Xulei
• Senger Alois

Classifications

• B66B29/005 Primary

  Applications of security monitors · CPC title

• G06V10/7788 Primary

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title

• G06F18/41

  Interactive pattern learning with a human teacher · CPC title

• B66B21/02

  Escalators · CPC title

• B66B29/08

  Means to facilitate passenger entry or exit (moving handrails B66B23/24) · CPC title