Machine learning-based defect detection of a specimen

Patent metadata

Field	Value
Publication number	US-11449711-B2
Application number	US-202016733219-A
Country	US
Kind code	B2
Filing date	Jan 2, 2020
Priority date	Jan 2, 2020
Publication date	Sep 20, 2022
Grant date	Sep 20, 2022

Abstract

There is provided a method of defect detection on a specimen and a system thereof. The method includes: obtaining a runtime image representative of at least a portion of the specimen; processing the runtime image using a supervised model to obtain a first output indicative of the estimated presence of first defects on the runtime image; processing the runtime image using an unsupervised model component to obtain a second output indicative of the estimated presence of second defects on the runtime image; and combining the first output and the second output using one or more optimized parameters to obtain a defect detection result of the specimen.

First claim

What is claimed is:

1. A computerized method for runtime defect detection on a specimen, the computerized method being performed by a processor and memory circuitry (PMC), the computerized method comprising:
obtaining a runtime image representative of at least a portion of the specimen;
processing, in a runtime phase, the runtime image using a supervised model component to obtain a first output indicative of estimated presence of first defects on the runtime image in the runtime phase, wherein the supervised model component is previously trained in a training phase prior to the runtime phase using a first training set comprising a plurality of first images each representative of the specimen and corresponding label data indicative of first defect distribution on the plurality of first images;
separately processing, in the runtime phase, the runtime image using an unsupervised model component to obtain a second output indicative of estimated presence of second defects on the runtime image in the runtime phase, wherein the unsupervised model component is previously trained in the training phase using a second training set including a plurality of second images each representative of the specimen, each second image of the plurality of second images being a defect-free reference image of a first image of the plurality of first images; and
combining the first output and the second output using one or more optimized parameters to obtain a runtime defect detection result of the specimen.

2. The computerized method according to claim 1 , wherein the one or more optimized parameters are obtained during training using a third training set.

3. The computerized method according to claim 2 , wherein:
the first output is a first grade map representative of estimated probabilities of the first defects on the runtime image, and the second output is a second grade map representative of estimated probabilities of the second defects on the runtime image;
the combining of the first output and the second output is performed using a segmentation model component operatively connected to the supervised model component and the unsupervised model component, to obtain a composite grade map indicative of estimated probabilities of the first defects and the second defects on the specimen; and
the segmentation model component is trained using the third training set based on outputs of the supervised model component and the unsupervised model component.

4. The computerized method according to claim 2 , wherein:
the first output is a first grade map representative of estimated probabilities of the first defects on the runtime image, and the second output is a second grade map representative of estimated probabilities of the second defects on the runtime image;
the combining of the first output and the second output comprises combining the first grade map and the second grade map with respective global weights to generate a composite grade map indicative of estimated probabilities of the first defects and the second defects on the specimen; and
the respective global weights are optimized during the training using the third training set.

5. The computerized method according to claim 2 , wherein:
the processing of the runtime image using the supervised model component comprises generating a first grade map representative of estimated probabilities of the first defects on the runtime image and applying a first threshold to the first grade map to obtain a first defect map;
the separately processing of the runtime image using the unsupervised model component comprises generating a second grade map representative of estimated probabilities of the second defects on the runtime image, and applying a second threshold to the second grade map to obtain a second defect map, the first threshold and the second threshold being optimized during the training using the third training set; and
the combining of the first output and the second output comprises combining the first defect map and the second defect map to generate a composite defect map.

6. The computerized method according to claim 4 , wherein the respective global weights are obtained using a non-gradient optimization function during the training using the third training set.

7. The computerized method according to claim 1 , wherein the supervised model component is trained by processing each first image of the plurality of first images to generate a corresponding first grade map representative of estimated probabilities of the first defects on the first image, and optimizing the supervised model component based on the label data corresponding to the first image.

8. The computerized method according to claim 1 , wherein the unsupervised model component is trained by processing each second image of the plurality of second images to generate a corresponding second grade map representative of estimated probabilities of the second defects on the plurality of second images, and optimizing the unsupervised model component based on the second grade map in relation to the plurality of second images.

9. The computerized method according to claim 1 , wherein the first training set further includes, for each first image of the plurality of first images, corresponding design data, and/or at least one reference image, and the obtaining further comprises obtaining at least one of design data or at least one reference image of the runtime image.

10. The computerized method according to claim 1 , wherein the second training set further includes, for each second image of the plurality of second images, corresponding design data, and the obtaining further comprises obtaining design data of the runtime image.

11. The computerized method according to claim 1 , wherein the supervised model component and the unsupervised model component are trained separately.

12. The computerized method according to claim 1 , further comprising obtaining, during runtime, one or more new first images each with label data indicative of presence of one or more new classes of defects, and retraining the supervised model component using the new first images.

13. The computerized method according to claim 1 , wherein the runtime image is a review image generated by a review tool.

14. The computerized method according to claim 1 , further comprising processing the runtime image using one or more additional components of supervised or unsupervised model components to obtain one or more additional outputs indicative of estimated presence of additional defects on the runtime image, wherein the one or more additional components are trained using one or more additional training sets including training images from at least one of different layers of the specimen or from different specimens.

15. A computerized system of runtime defect detection on a specimen, the computerized system comprising a processor and memory circuitry (PMC) configured to:
obtain a runtime image representative of at least a portion of the specimen;
process, in a runtime phase, the runtime image using a supervised model component to obtain a first output indicative of estimated presence of first defects on the runtime image in the runtime phase, wherein the supervised model component is previously trained in a training phase prior to the runtime phase using a first training set comprising a plurality of first images each representative of the specimen and corresponding label data indicative of first defect distribution on the plurality of first images;
separately process, in the runtime phase, the runtime image using an unsupervised model component to obtain a second output indi

Assignees

• Applied Materials Israel Ltd
• Applied Mat Isreal Ltd

Inventors

• Badanes Ran
• Schleyen Ran
• Cohen Boaz
• Peleg Irad
• Suhanov Denis
• Shtalrid Ore

Classifications

• G06T7/0004 Primary

  Industrial image inspection · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/809

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title