Diagnostic systems and methods for deep learning models configured for semiconductor applications

Patent metadata

Field	Value
Publication number	US-11580398-B2
Application number	US-201715694719-A
Country	US
Kind code	B2
Filing date	Sep 1, 2017
Priority date	Oct 14, 2016
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

Methods and systems for performing diagnostic functions for a deep learning model are provided. One system includes one or more components executed by one or more computer subsystems. The one or more components include a deep learning model configured for determining information from an image generated for a specimen by an imaging tool. The one or more components also include a diagnostic component configured for determining one or more causal portions of the image that resulted in the information being determined and for performing one or more functions based on the determined one or more causal portions of the image.

First claim

What is claimed is:

1. A system configured to perform diagnostic functions for a deep learning model, comprising:
one or more computer subsystems having one or more processors that execute instructions from a memory medium; and
one or more components executed by the one or more computer subsystems and stored on a non-transitory computer-readable medium, wherein the one or more components comprise:
a deep learning model configured for determining information from an image generated for a specimen by an imaging tool;
a diagnostic component configured for determining one or more causal portions of the image that resulted in the information being determined and for performing one or more functions based on the determined one or more causal portions of the image, wherein the one or more causal portions are determined by using causal back propagation to assign causal scores to pixels in an image space corresponding to the image, wherein the one or more functions comprise determining one or more characteristics of the one or more causal portions, and wherein determining the one or more characteristics comprises qualitatively and quantitatively identifying an importance of each pixel of the image input to the deep learning model in contributing to said determining the information;
a visualization component configured for displaying at least the image, the determined information, and the determined one or more causal portions to a user; and
a user interface component configured to receive input from the user after said displaying, wherein the input comprises correct one or more causal portions received from the user after the image, the determined information, and the determined one or more causal portions have been displayed to the user, wherein at least one of the one or more functions comprises determining if the one or more causal portions that resulted in the information being determined are the correct one or more causal portions of the image, wherein the one or more functions further comprise generating an augmented image based on the image and a result of the determining if the one or more causal portions that resulted in the information being determined are the correct one or more causal portions of the image, and wherein the one or more computer subsystems are configured for further training the deep learning model using the augmented image.

2. The system of claim 1 , wherein the information comprises a classification for a defect detected on the specimen.

3. The system of claim 1 , wherein the information comprises features of the image extracted by the deep learning model.

4. The system of claim 1 , wherein the information comprises a simulated image generated from the image.

5. The system of claim 1 , wherein the information comprises one or more segmentation regions generated from the image.

6. The system of claim 1 , wherein the information comprises a multi-dimensional output generated from the image.

7. The system of claim 1 , wherein the deep learning model is a trained deep learning model.

8. The system of claim 1 , wherein the deep learning model is further configured as a neural network.

9. The system of claim 1 , wherein the one or more functions further comprise altering one or more parameters of the deep learning model based on the determined one or more causal portions.

10. The system of claim 1 , wherein the one or more functions performed by the diagnostic component are determined based on the input from the user or input from an additional user.

11. The system of claim 1 , wherein the diagnostic component is further configured for determining the one or more causal portions by computing a local sensitivity.

12. The system of claim 1 , wherein the causal back propagation is performed using a deconvolution heatmap algorithm.

13. The system of claim 1 , wherein the causal back propagation is performed using a layer-wise relevance propagation.

14. The system of claim 1 , wherein the causal back propagation is performed using a deep lift algorithm.

15. The system of claim 1 , wherein the diagnostic component is further configured for determining the one or more causal portions by global average pooling.

16. The system of claim 1 , wherein the diagnostic component is further configured for determining the one or more causal portions by computing a path integral on gradients.

17. The system of claim 1 , wherein the diagnostic component is further configured for determining the one or more causal portions by computing a partial dependence plot.

18. The system of claim 1 , wherein the diagnostic component is further configured for determining the one or more causal portions by computing a partial dependence plot with path integral.

19. The system of claim 1 , wherein the one or more functions further comprise determining, based on the one or more characteristics of the one or more causal portions, if additional images for the specimen should be collected from the imaging tool and used for additional training of the deep learning model.

20. The system of claim 1 , wherein the one or more functions further comprise generating a data augmentation method for application to additional images input to the deep learning model.

21. The system of claim 1 , wherein the one or more functions further comprise identifying the one or more causal portions as one or more regions of interest in the image and tuning the deep learning model based on the one or more regions of interest.

22. The system of claim 1 , wherein the one or more functions further comprise identifying the one or more causal portions as one or more regions of interest in the image and training an additional deep learning model based on the one or more regions of interest.

23. The system of claim 1 , wherein the imaging tool is configured as an inspection tool.

24. The system of claim 1 , wherein the imaging tool is configured as a metrology tool.

25. The system of claim 1 , wherein the imaging tool is configured as an electron beam based imaging tool.

26. The system of claim 1 , wherein the imaging tool is configured as an optical based imaging tool.

27. The system of claim 1 , wherein the specimen is a wafer.

28. The system of claim 1 , wherein the specimen is a reticle.

29. A system configured to perform diagnostic functions for a deep learning model, comprising:
an imaging tool configured for generating images of a specimen;
one or more computer subsystems configured for acquiring the images; and
one or more components executed by the one or more computer subsystems, wherein the one or more components comprise:
a deep learning model configured for determining information from an image generated for the specimen by the imaging tool;
a diagnostic component configured for determining one or more causal portions of the image that resulted in the information being determined and for performing one or more functions based on the determined one or more causal portions of the image, wherein the one or more causal portions are determined by using causal back propagation to assign causal scores to pixels in an image space corresponding to the image, wherein the one or more functions comprise determining one or more characteristics of the one or more causal portions, and wherein determining the one or more characteristics comprises qualitatively and quantitatively identifying an im

Assignees

• Kla Tencor Corp
• Kla Tenor Corp

Inventors

• Zhang Jing
• Donapati Ravi Chandra
• Roulo Mark
• Bhaskar Kris

Classifications

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/40

  Extraction of image or video features · CPC title