Apparatus and method with defect detection

Patent metadata

Field	Value
Publication number	US-2025225647-A1
Application number	US-202418625556-A
Country	US
Kind code	A1
Filing date	Apr 3, 2024
Priority date	Oct 23, 2023
Publication date	Jul 10, 2025
Grant date	—

Abstract

A processor-implemented defect detection method are provided. The defect detection method includes generating a plurality of text data by adding a plurality of candidate classes which indicate whether a product is defective to product text information; and detecting whether a product image represents a defective product using an image model provided the product image, and a text model provided the plurality of text data.

First claim

What is claimed is:

1 . A processor-implemented defect detecting method, the method comprising:
generating a plurality of text data by adding a plurality of candidate classes, which indicate whether a product is defective, to product text information; and detecting whether a product image represents a defective product using an image model provided the product image, and a text model provided the plurality of text data.

2 . The method of claim 1 , wherein the detecting of whether the product image represents a defective product comprises:
calculating respective similarities between a first feature map generated by the image model and a plurality of second feature maps generated by the text model; and determining whether the product is defective based on the calculated respective similarities.

3 . The method of claim 2 , wherein the determining of whether the product represents a defective product comprises:
calculating respective similarities between each of the plurality of second feature maps and the first feature map; converting the respective similarities between each of the plurality of second feature maps and the first feature map into corresponding scores; and outputting a candidate class comprised in text data corresponding to a highest score among the corresponding scores.

4 . The method of claim 1 , wherein the product text information comprises at least one of customer company information, production area information for the product, factory information for the product, product line information for the product, process information for the product, external environment information for the product, and inspection surface information for the product.

5 . The method of claim 1 , wherein each of the plurality of candidate classes comprise information indicating whether the product is defective, and a defect type of the product.

6 . The method of claim 1 wherein the product text information comprises a plurality of information that identifies the product, and wherein the plurality of information and the candidate class of the plurality of candidate classes included in each text data are distinguished by a special character.

7 . The method of claim 1 , further comprising:
training the image model and the text model using a plurality of training data, wherein each of the plurality of training data comprises a pair that includes text data and a product image, and the training text data comprises product text information and a ground truth label.

8 . The method of claim 7 , wherein the training of the image model and the text model comprises:
calculating a similarity between a third feature map for the trained product image output from the image model and a fourth feature map for the training text data output from the text model; and training the image model and the text model to increase the similarity between the third feature map and the fourth feature map through plural training iterations.

9 . An apparatus comprising:
one or more processors configured to:
generate a plurality of text data by combining each of a plurality of candidate classes with product text information;
generate a first feature map using an image model based on a product image; generate a plurality of second feature maps using a text model based on the plurality of text data; and detect whether the product image represents a defective product based on a determined similarity between each of the plurality of second feature maps and the first feature map.

10 . The apparatus of claim 9 , wherein the product text information comprises at least one of customer company information, production area information for the product, factory information for the product, product line information for the product, process information for the product, external environment information for the product, and inspection surface information for the product.

11 . The apparatus of claim 10 ,
wherein the customer company information comprises defect inspection standard information of a customer company, and wherein the external environment information of the product comprises lighting condition information.

12 . The apparatus of claim 9 , wherein each of the plurality of candidate classes comprise information indicating whether the product is defective, and a defect type of the product.

13 . The apparatus of claim 9 , wherein the classifier is configured to convert the similarity between each of the plurality of second feature maps and the first feature map into a score based on a softmax function, and output a candidate class corresponding to a highest score among the converted score as a result of whether the product image is defective.

14 . The apparatus of claim 9 , wherein the one or more processors are further configured to:
train the image model and the text model using a plurality of training data consisting of pairs of respective training text data and training product image through a minimization of a loss calculated from a similarity between a third feature map for the training product image output from the image model and a fourth feature map for the training text data output from the text model, respectively, wherein the training text data comprises the product text information and a ground truth label.

15 . The apparatus of claim 9 , wherein the one or more processors are further configured to determine a plurality of candidate classes based on decision information.

16 . The apparatus of claim 15 , wherein the decision information comprises defect type information and defect status information.

17 . The apparatus of claim 16 , wherein the user-specific product text information comprises at least one of customer company information, production area information, factory information, product line information, process information, external environment information, and inspection surface information.

18 . The apparatus of claim 17 , wherein the customer company information comprises defect inspection standard information of a customer company.

19 . An apparatus, comprising:
processors configured to:
train an image model, using product text data and product images, to generate a first feature map;
train a text model, using the product text data, to generate a second feature map; and
train a classifier to convert a determined similarity between the first feature map and the second feature map into a class score that is indicative of whether a product is defective,
wherein the image model, the text model, and the classifier are trained together.

20 . The apparatus of claim 19 , wherein the product text data comprises at least one of customer company information, production area information for the product, factory information for the product, product line information for the product, process information for the product, external environment information for the product, and inspection surface information for the product.

Assignees

• Samsung Electro Mech

Inventors

• Park Dongjin
• Jeong Daun
• Cho Hansang

Classifications

• G06T7/001 Primary

  using an image reference approach · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G01N2021/8887

  based on image processing techniques · CPC title

• G01N21/8851

  Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges (G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002) · CPC title