Detection method, electronic device and non-transitory computer-readable storage medium

Patent metadata

Field	Value
Publication number	US-12236580-B2
Application number	US-202318543121-A
Country	US
Kind code	B2
Filing date	Dec 18, 2023
Priority date	May 29, 2020
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

The present disclosure provides a detection method. The detection method includes inputting an image to be detected into a detection model being pre-constructed and detecting the image to be detected. The detection model includes a defect classification identification sub-model configured to identify a classification of a defect in the image to be detected, and the defect classification identification sub-model comprises a plurality of base models and a secondary model. The present disclosure further provides an electronic device and a non-transitory computer-readable storage medium.

First claim

What is claimed is:

1. A detection method, comprising:
inputting an image to be detected into a detection model being pre-constructed and detecting the image to be detected;
wherein the detection model comprises:
a defect classification identification sub-model configured to identify a classification of a defect in the image to be detected; wherein the defect classification identification sub-model comprises a plurality of base models and a secondary model;
the plurality of base models are configured to respectively determine an initial classification of the defect in the image to be detected; and
the secondary model is configured to determine a final classification of the defect in the image to be detected according to input data obtained by integrating output data of the plurality of base models;
wherein the detection model further comprises: a defect position identification sub-model configured to mark a position of the defect in the image to be detected;
wherein the defect position identification sub-model is an object detector; and
wherein the defect position identification sub-model is obtained by training the object detector based on an original data set;
the training the object detector based on the original data set comprises:
acquiring the original data set comprising a plurality of images to be detected with known defects; and
classifying first regions corresponding to defects of all classifications of defects in the plurality of images to be detected with known defects into one classification as a foreground and classifying regions outside the first regions in the plurality of images to be detected with known defects into the other classification as a background such that the object detector only distinguishes between the foreground and the background when the object detector is trained, to identify only positions of the defects.

2. The detection method of claim 1 , wherein the plurality of base models are obtained by respectively training a same Convolutional Neural Network model with a plurality of first training data sets satisfying different probability distributions.

3. The detection method of claim 2 , further comprising generating the plurality of first training data sets, which comprises:
generating an original data set comprising a plurality of images to be detected with known defects;
respectively determining sampling ratios of images to be detected of different classifications of defects corresponding to a plurality of probability distributions; and
respectively sampling the original data set according to the sampling ratios of the images to be detected of different classifications of defects to obtain the plurality of first training data sets.

4. The detection method of claim 2 , wherein the Convolutional Neural Network model comprises a fully-connected layer, a supplementary convolution layer, a batch normalization layer and a random discard layer;
the supplementary convolution layer is configured to convolve data to be input into the fully-connected layer such that the data convolved by the supplementary convolution layer meets an input dimension of the fully-connected layer;
the batch normalization layer is configured to normalize the data to be input into the fully-connected layer;
the random discard layer is configured to randomly discard a part of neural network units of the Convolutional Neural Network model to avoid overfitting; and
when the Convolutional Neural Network model is trained, the fully-connected layer is initialized with a first algorithm, the fully-connected layer is regularized with a second algorithm, and the supplementary convolution layer is initialized with a third algorithm.

5. The detection method of claim 1 , wherein the secondary model is a classifier comprising a plurality of fully-connected layers and a normalized exponential function layer.

6. The detection method of claim 1 , wherein before the classifying first regions corresponding to defects of all classifications of defects in the plurality of images to be detected with known defects into one classification as a foreground and classifying regions outside the first regions in the plurality of images to be detected with known defects into the other classification as a background, the training the object detector based on the original data set further comprises: removing an image to be detected with a classification of a normal image, a black image or a fuzzy image in the original data set.

7. An electronic device, comprising:
one or more processors;
a memory, storing one or more programs, which when executed by the one or more processors, cause the one or more processors to perform a detection method; and
one or more I/O interfaces connected between the one or more processors and the memory and configured to exchange information between the one re more processors and the memory, wherein
the detection method comprises:
inputting an image to be detected into a detection model being pre-constructed and detecting the image to be detected;
wherein the detection model comprises:
a defect classification identification sub-model configured to identify a classification of a defect in the image to be detected;
wherein the defect classification identification sub-model comprises a plurality of base models and a secondary model;
the plurality of base models are configured to respectively determine an initial classification of the defect in the image to be detected; and
the secondary model is configured to determine a final classification of the defect in the image to be detected according to input data obtained by integrating output data of the plurality of base models;
wherein the detection model further comprises: a defect position identification sub-model configured to mark a position of the defect in the image to be detected;
wherein the defect position identification sub-model is an object detector; and
wherein the defect position identification sub-model is obtained by training the object detector based on an original data set;
the training the object detector based on the original data set comprises:
acquiring the original data set comprising a plurality of images to be detected with known defects; and
classifying first regions corresponding to defects of all classifications of defects in the plurality of images to be detected with known defects into one classification as a foreground and classifying regions outside the first regions in the plurality of images to be detected with known defects into the other classification as a background such that the object detector only distinguishes between the foreground and the background when the object detector is trained, to identify only positions of the defects.

8. The electronic device of claim 7 , wherein the plurality of base models are obtained by respectively training a same Convolutional Neural Network model with a plurality of first training data sets satisfying different probability distributions.

9. The electronic device of claim 8 , wherein comprising generating the plurality of first training data sets, which comprises:
generating an original data set comprising a plurality of images to be detected with known defects;
respectively determining sampling ratios of images to be detected of different classifications of defects corresponding to a plurality of probability distributions; and
respectively sampling the original data set according to the sampling ratios of the images to be detected of different classifications of defects to obtain the plurality of first training data sets.

10. The electronic device of claim 8 , wherein the Convolutional Neural Network model comprises a fully-connected layer, a supplementary

Assignees

• Boe Technology Group Co Ltd

Inventors

• Liu Yongzhang
• Li Zhaoyue
• Chai Dong
• Wang Hong

Classifications

• G01N29/4445

  Classification of defects · CPC title

• G06T2207/30121

  CRT, LCD or plasma display · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title