System and method for determining situation of facility by imaging sensing data of facility

Patent metadata

Field	Value
Publication number	US-11580629-B2
Application number	US-202017013859-A
Country	US
Kind code	B2
Filing date	Sep 8, 2020
Priority date	Oct 25, 2019
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

Embodiments relate to a method and system for determining a situation of a facility by imaging a sensing data of the facility including receiving sensing data through a plurality of sensors at a query time, generating a situation image at the query time, showing the situation of the facility at the query time based on the sensing data, and determining if an abnormal situation occurred at the query time by applying the situation image to a pre-learned situation determination model.

First claim

What is claimed is:

1. A method for determining a situation of a facility by imaging a sensing data of the facility, performed by a computing device including a processor, the method comprising:
receiving sensing data through a plurality of sensors at a query time;
generating a situation image at the query time, showing the situation of the facility at the query time based on the sensing data, wherein generating the situation image comprises:
arranging the sensing data of each sensor at the query time; and
forming an N*N pixel set, wherein N is a total number of sensors in the plurality of sensors and each pixel is associated with a first sensor and a second sensor; and

determining if an abnormal situation occurred at the query time by applying the situation image to a pre-learned situation determination model.

2. The method according to claim 1 , wherein the plurality of sensors has a plurality of types, and
the sensing data is multivariate time-series data.

3. The method according to claim 1 , further comprising:
preprocessing to normalize the sensing data before generating the situation image.

4. The method according to claim 1 , wherein arranging the sensing data of each sensor at the query time comprises:
arranging according to a pre-stored sensor sequence number, and
the sensor sequence number is in accordance with an operating sequence of equipment in the facility.

5. The method according to claim 1 , wherein generating the situation image comprises:
calculating a color value of a pixel at the query time, based on a difference between first sensing data and second sensing data at the query time, received through a first sensor and a second sensor associated with the pixel;
searching for a color corresponding to the calculated color value in a pre-stored color table; and
assigning the color corresponding to the calculated color value as the color of the pixel.

6. The method according to claim 5 , wherein calculating the color value of the pixel at the query time further comprises:
calculating a difference value of the pixel for each prior query time based on a magnitude difference between the first sensing data and second sensing data during a predetermined time period prior to the query time;
applying a time weight for each prior query time to the difference value for each time, respectively; and
calculating, as the color value of the pixel at the query time, a time-weighted difference value of the pixel based on the difference values calculated during the predetermined time period and their respective applied time weights.

7. The method according to claim 6 , wherein the time weight for each time has a higher value as it is closer to the query time.

8. The method according to claim 1 , wherein determining if the abnormal situation occurred at the query time comprises:
generating a situation secondary image by applying the situation image to a pre-learned situation determination model;
calculating an anomaly score at the query time based on the situation image and the situation secondary image; and
determining that the abnormal situation occurred at the query time when the anomaly score is higher than a preset threshold.

9. The method according to claim 8 , wherein the situation determination model is trained to generate output data having a same data distribution as the data distribution of training samples used to train the situation determination model or having a minimum reconstruction error with respect to the data distribution of the training samples.

10. The method according to claim 9 , wherein the training samples used for training include sensing data of a normal situation.

11. The method according to claim 8 , wherein the situation determination model generates the situation secondary image having a smallest vector distance with respect to a situation image showing a normal situation.

12. The method according to claim 1 , further comprising:
detecting the sensor having sensed the abnormal situation based on the situation image, when it is determined that the abnormal situation occurred.

13. The method according to claim 12 , wherein detecting the sensor comprises:
generating a residual image at the query time based on the situation image and a situation secondary image generated by the situation determination model;
determining a pixel having a larger color value than a preset residual threshold based on the color value of the pixel included in the residual image; and
determining each sequence number of a plurality of sensors associated with the determined pixel by a coordinate of the determined pixel, and
the residual threshold is larger than the color value of the pixel within the residual image generated based on the sensing data of the normal situation.

14. A non-transitory computer-readable medium for storing program instructions that can be read by a computing device and executed by the computing device, wherein when the program instructions are executed by a processor of the computing device, the program instructions enable the processor to perform the steps:
receiving sensing data through a plurality of sensors at a query time;
generating a situation image at the query time, showing the situation of the facility at the query time based on the sensing data, wherein generating the situation image comprises:
arranging the sensing data of each sensor at the query time; and
forming an N*N pixel set, wherein N is a total number of sensors in the plurality of sensors and each pixel is associated with a first sensor and a second sensor; and

determining if an abnormal situation occurred at the query time by applying the situation image to a pre-learned situation determination model.

15. A system for determining a situation of a facility by imaging a sensing data of the facility, the system comprising:
a plurality of sensors installed in the facility;
a receiving device to receive sensing data through the plurality of sensors;
an image conversion unit to generate a situation image showing the situation of the facility at the query time based on the sensing data, wherein the image conversion unit is configured to generate the situation image by:
arranging the sensing data of each sensor at the query time; and
forming an N*N pixel set, wherein N is a total number of sensors in the plurality of sensors and each pixel is associated with a first sensor and a second sensor;

a situation determination unit to determine if an abnormal situation occurred at the query time by applying the situation image to a pre-learned situation determination model; and
an anomaly location detection unit to detect the sensor having sensed an abnormal situation when the situation of at least a part of the facility is determined to be the abnormal situation.

16. The system according to claim 15 , wherein the situation determination model is trained to generate output data having a same data distribution as the data distribution of the training sample sed to train the situation determination model or having a minimum reconstruction error with respect to the data distribution of the training samples.

17. The system according to claim 15 , wherein the situation determination model generates a situation secondary image having a smallest vector distance with respect to a situation image showing a normal situation.

18. The system according to claim 15 , wherein the anomaly location detection unit is configured to:
generate a residual image at the query time based on the situation image and a situation secondary image gen

Assignees

• Korea Inst Sci & Tech

Inventors

• Kim Ig Jae
• Choi Heeseung
• Lim Hyunki
• Choi Yeji

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title