Integration of automatic and manual defect classification

Patent metadata

Field	Value
Publication number	US-10043264-B2
Application number	US-201213451496-A
Country	US
Kind code	B2
Filing date	Apr 19, 2012
Priority date	Apr 19, 2012
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

A method for defect classification includes storing definitions of defect classes in terms of a classification rules in a multi-dimensional feature space. Inspection data associated with defects detected in one or more samples under inspection is received. A plurality of first classification results is generated by applying an automatic classifier to the inspection data based on the definitions, the plurality of first classification results comprising a class label and a corresponding confidence level for a defect. Upon determining that a confidence level for a defect is below a predetermined confidence threshold, a plurality of second classification results are generated by applying at least one inspection modality to the defect. A report is generated comprising a distribution of the defects among the defect classes by combining the plurality of first classification results and the plurality of second classification results.

First claim

What is claimed is:

1. A method for defect classification comprising:
storing, by a processor, a plurality of definitions of a plurality of defect classes in terms of a plurality of classification rules in a multi-dimensional feature space, wherein the plurality of classification rules, for each given defect class, defines in a feature space a boundary of a region associated with the given class and provides a confidence measure associated with classification of a defect to the given defect class, the confidence measure being indicative of a level of confidence as a function of the location of the defect in the feature space with respect to the respective boundaries;
receiving, by the processor, inspection data associated with a plurality of defects detected in one or more samples under inspection;
receiving, by the processor and from an operator, a classification performance measure selected from a plurality of performance measures, wherein the plurality of performance measures comprises at least one of a maximum rejection rate or a target purity level;
determining at least one confidence threshold corresponding to the classification performance measure;
applying, by the processor, an automatic classifier to the inspection data, the automatic classifier based on the plurality of definitions, and identifying a plurality of defects each classified with a low level of confidence based on the at least one confidence threshold and indicative of the defect being located in an overlap region between the respective boundaries of at least two of the defect classes;
generating, by the processor, a plurality of classification results by applying, to the identified plurality of defects classified with the low level of confidence, at least one inspection modality that is different than the automatic classifier to assign each of the identified plurality of defects to one of the at least two of the defect classes associated with the overlap region;
refining, by the processor, the automatic classifier to adjust boundaries of one or more defect classes of the plurality of classes when a threshold amount of the identified plurality of defects located in the overlap region have been classified by the at least one inspection modality that is different than the automatic classifier, wherein the refining is provided by training the automatic classifier using each classification result of the plurality of classification results of the identified plurality of defects classified with the low level of confidence.

2. The method of claim 1 , further comprising:
determining to each given defect class a confidence threshold of the given defect class, wherein a confidence threshold of the given defect class is based on the loci of the respective boundary in the feature space.

3. The method of claim 2 , wherein an extent of a boundary in the feature space is controlled by a confidence level of a respective defect class.

4. The method of claim 2 , wherein the plurality of definitions of the plurality of defect classes comprises a kernel function having a parameter, and wherein applying the automatic classifier to the inspection data comprises selecting a value of the parameter for the plurality of defect classes based on the at least one confidence threshold.

5. The method of claim 2 , wherein at least two defect classes have differently determined confidence thresholds.

6. The method of claim 1 , wherein the at least one inspection modality that is different than the automatic classifier corresponds to a visual inspection, and wherein applying the automatic classifier to the inspection data comprises applying a multi-class classifier to the inspection data to assign defects of the plurality of defects to a defect class of the plurality of defect classes, wherein the multi-class classifier is configured to identify the defects in the overlap region.

7. An apparatus comprising:
a memory to store a plurality of definitions of a plurality of defect classes in terms of a plurality of classification rules in a multi-dimensional feature space, wherein the plurality of classification rules, for each given defect class, defines in a feature space a boundary of a region associated with the given class and provides a confidence measure associated with classification of a defect to the given defect class, the confidence measure being indicative of a level of confidence as a function of the location of the defect in the feature space with respect to the respective boundaries; and
a processor, operatively coupled with the memory, to:
receive inspection data associated with a plurality of defects detected in one or more samples under inspection;
receive, from an operator, a classification performance measure selected from a plurality of performance measures, wherein the plurality of performance measures comprises at least one of a maximum rejection rate or a target purity level;
determine at least one confidence threshold corresponding to the classification performance measure;
apply an automatic classifier to the inspection data, the automatic classifier based on the plurality of definitions, and identifying a plurality of defects each classified with a low level of confidence based on the at least one confidence threshold and indicative of the defect being located in an overlap region between the respective boundaries of at least two of the defect classes;
generate a plurality of classification results by applying, to the identified plurality of defects classified with the low level of confidence, at least one inspection modality that is different than the automatic classifier to assign each of the identified plurality of defects to one of the at least two of the defect classes associated with the overlap region; and
refine the automatic classifier to adjust boundaries of one or more defect classes of the plurality of classes when a threshold amount of the identified plurality of defects located in the overlap region have been classified by the at least one inspection modality that is different than the automatic classifier, wherein the refining is provided by training the automatic classifier using each classification result of the plurality of classification results of the identified plurality of defects classified with the low level of confidence.

8. The apparatus of claim 7 , wherein the processor is further to:
determine to each given defect class a confidence threshold of the given defect class, wherein a confidence threshold of the given defect class is based on the loci of the respective boundary in the feature space.

9. The apparatus of claim 8 , wherein an extent of a boundary in the feature space is controlled by a confidence level of a respective defect class.

10. The apparatus of claim 8 , wherein the plurality of definitions of the plurality of defect classes comprises a kernel function having a parameter, and wherein applying the automatic classifier to the inspection data comprises selecting a value of the parameter for the plurality of defect classes based on the at least one confidence threshold.

11. The apparatus of claim 8 , wherein at least two defect classes have differently determined confidence thresholds.

12. The apparatus of claim 7 , wherein the at least one inspection modality that is different than the automatic classifier corresponds to a visual inspection, and wherein applying the automatic classifier to the inspection data comprises applying a multi-class classifier to the inspection data to assign defects of the plurality of defects to a defect class of the plurality of defect classes, wherein the multi-class classifier is configured to identify the defects in the overlap region.

Assignees

• Greenberg Gadi
• Kaizerman Idan
• Rozenman Efrat
• Applied Materials Israel Ltd

Inventors

• Greenberg Gadi
• Kaizerman Idan
• Rozenman Efrat

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2433

  Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06V10/987

  with the intervention of an operator · CPC title

• G06K9/033

  Physics · mapped topic