Defective pixel correction using adversarial networks

Patent metadata

Field	Value
Publication number	US-10825149-B2
Application number	US-201816110282-A
Country	US
Kind code	B2
Filing date	Aug 23, 2018
Priority date	Aug 23, 2018
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

A framework for defective pixel correction using adversarial networks. In accordance with one aspect, the framework receives first and second training image datasets. The framework performs adversarial training of a corrector and a classifier with the first and second training image datasets respectively. The corrector may be trained to correct a first input image and the classifier may be trained to recognize whether a second input image is real or generated by the corrector. The framework applies the trained corrector to a current image to correct any defective pixels and generate a corrected image. The corrected image may then be presented.

First claim

What is claimed is:

1. One or more non-transitory computer readable media embodying a program of instructions executable by machine to perform operations for defective pixel correction, the operations comprising:
receiving first and second training image datasets;
performing adversarial training of a corrector and a classifier with the first and second training image datasets respectively, wherein the corrector comprises a defective pixel detector subnetwork and a raw correction subnetwork, wherein the defective pixel detector subnetwork is trained to encode one or more locations of one or more defective pixels of a first input image, wherein the raw correction subnetwork is trained to correct the one or more defective pixels in the first input image, wherein the classifier is trained to recognize whether a second input image is real or generated by the corrector;
applying the trained corrector to a current image to generate a corrected image; and
presenting the corrected image.

2. The one or more non-transitory computer readable media of claim 1 wherein the current image comprises a two-dimensional X-ray image.

3. The one or more non-transitory computer readable media of claim 1 wherein the corrector and the classifier comprise two-dimensional deep convolutional networks.

4. The one or more non-transitory computer readable media of claim 1 wherein the corrector further comprises an encoder subnetwork that provides a set of feature channels based on the first input image to the defective pixel detector subnetwork and the raw correction subnetwork.

5. A system comprising:
a non-transitory memory device for storing computer readable program code; and
a processor in communication with the memory device, the processor being operative with the computer readable program code to perform operations including
receiving first and second training image datasets,
performing adversarial training of a corrector and a classifier with the first and second training image datasets respectively, wherein the corrector comprises a defective pixel detector subnetwork and a raw correction subnetwork, wherein the defective pixel detector subnetwork is trained to encode one or more locations of one or more defective pixels of a first input image, wherein the raw correction subnetwork is trained to correct the one or more defective pixels in the first input image, wherein classifier is trained to recognize whether a second input image is real or generated by the corrector,
applying the trained corrector to a current image to generate a corrected image, and
presenting the corrected image.

6. The system of claim 5 wherein the first training image dataset comprises images associated with segmentation masks delineating defective pixels.

7. The system of claim 5 wherein the second training image dataset comprises real images acquired directly from an imaging device and synthetic images generated by the corrector.

8. The system of claim 5 wherein the corrector comprises a deep convolutional network.

9. The system of claim 5 wherein the corrector further comprises an encoder subnetwork that provides a set of feature channels based on the first input image to the defective pixel detector subnetwork and the raw correction subnetwork.

10. The system of claim 9 wherein the corrected image is a linear combination of the first input image and an output of the raw correction subnetwork weighted by a probabilistic output of the defective pixel detector subnetwork.

11. The system of claim 9 wherein the corrector further comprises a fusion subnetwork that outputs the corrected image.

12. The system of claim 11 wherein the fusion subnetwork takes as input the first input image, an output of the defective pixel detector subnetwork and an output of the raw correction subnetwork.

13. The system of claim 5 wherein the classifier comprises a deep convolutional network.

14. The system of claim 13 wherein the deep convolutional network comprises two-dimensional convolutional blocks.

15. The system of claim 5 wherein the processor is operative with the computer readable program code to simultaneously train the corrector and the classifier to minimize a corrector loss function and a classifier loss function.

16. The system of claim 15 wherein the classifier loss function comprises a binary cross-entropy function.

17. The system of claim 15 wherein the corrector loss function comprises a weighted summation of a difference term, a segmentation term and an adversarial term.

18. The system of claim 5 wherein the processor is operative with the computer readable program code to perform adversarial training by training the classifier and the corrector simultaneously for multiple epochs, wherein during one of the epochs, the classifier is iteratively trained given a batch of real and synthetic images and the corrector is iteratively trained using a current state of the classifier.

19. A method, comprising:
receiving first and second training image datasets;
performing adversarial training of a corrector and a classifier with the first and second training image datasets respectively, wherein the corrector comprises a defective pixel detector subnetwork and a raw correction subnetwork, wherein the defective pixel detector subnetwork is trained to encode one or more locations of one or more defective pixels of a first input image, wherein the raw correction subnetwork is trained to correct the one or more defective pixels in the first input image, wherein the classifier is trained to recognize whether a second input image is real or generated by the corrector;
applying the trained corrector to a current image to generate a corrected image; and
presenting the corrected image.

20. The method of claim 19 wherein applying the trained corrector to the current image further comprises generating a probabilistic map representing the defective pixels.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Schafer Sebastian
• Royalty Kevin
• Pauly Olivier

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/0014

  using an image reference approach · CPC title

• A61B6/5258

  involving detection or reduction of artifacts or noise · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title