Automated skin lesion segmentation using deep side layers

Patent metadata

Field	Value
Publication number	US-10373312-B2
Application number	US-201715642717-A
Country	US
Kind code	B2
Filing date	Jul 6, 2017
Priority date	Nov 6, 2016
Publication date	Aug 6, 2019
Grant date	Aug 6, 2019

Abstract

A method for computer-aided diagnosis of skin lesions includes obtaining a dermoscopic image, convolving the dermoscopic image in a plurality of convolutional layers, obtaining deconvolved outputs of at least two convolutional layers of the plurality of convolutional layers, obtaining side-output feature maps by applying loss functions to the deconvolved outputs of the at least two convolutional layers, obtaining a first concatenated feature map by concatenating the side-output feature maps with different first weights, obtaining a second concatenated feature map by concatenating the side-output feature maps with different second weights, and producing a final score map by convolving the first and second concatenated feature maps in a final convolutional layer followed by a loss layer. Also disclosed: a computer-readable medium embodying instructions for the method, and an apparatus configured to implement the method.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining a dermoscopic image;
convolving the dermoscopic image in a plurality of convolutional layers;
obtaining deconvolved outputs of at least two convolutional layers of the plurality of convolutional layers;
obtaining side-output feature maps by applying loss functions to the deconvolved outputs of the at least two convolutional layers;
obtaining a first concatenated feature map by concatenating the side-output feature maps with different first weights;
obtaining a second concatenated feature map by concatenating the side-output feature maps with different second weights; and
producing a final score map by convolving the first and second concatenated feature maps in a final convolutional layer followed by a final loss layer.

2. The method of claim 1 wherein the final loss layer directly notifies each side-output layer about the final objective of segmenting the skin lesion.

3. The method of claim 1 further comprising:
obtaining a training dermoscopic image and a training final score map;
convolving the training dermoscopic image in the plurality of convolutional layers;
deconvolving the outputs of the at least two convolutional layers of the plurality of convolutional layers;
obtaining trial side-output feature maps by applying loss functions to the deconvolved outputs of the at least two convolutional layers;
obtaining a trial concatenated feature map by concatenating the trial side-output feature maps with different weights;
convolving the trial concatenated feature map in the final convolutional layer followed by the final loss layer to produce a trial final score map;
assessing variances of the trial final score map from the training final score map;
adjusting filters of the plurality of convolutional layers in response to the variances; and
repeating the preceding steps until the variances are less than a threshold variance vector.

4. The method of claim 3 further comprising adjusting deconvolution filters in response to the variances.

5. The method of claim 1 wherein the plurality of convolutional layers comprise a VGG-16 neural network.

6. The method of claim 5 wherein the at least two convolutional layers include conv2_2, conv3_3, conv4_3, and conv5_3 layers of the VGG-16 neural network.

7. The method of claim 5 wherein the plurality of convolutional layers comprise a conv5_4 layer of dimensions 14×14×512, in addition to the layers of the VGG-16 neural network, and the conv5_4 layer is one of the at least two convolutional layers.

8. A non-transitory computer readable medium embodying computer executable instructions which when executed by a computer cause the computer to perform the method of:
obtaining a dermoscopic image;
convolving the dermoscopic image in a plurality of convolutional layers;
obtaining deconvolved outputs of at least two convolutional layers of the plurality of convolutional layers;
obtaining side-output feature maps by applying loss functions to the deconvolved outputs of the at least two convolutional layers;
obtaining a first concatenated feature map by concatenating the side-output feature maps with different first weights;
obtaining a second concatenated feature map by concatenating the side-output feature maps with different second weights; and
producing a final score map by convolving the first and second concatenated feature maps in a final convolutional layer followed by a loss layer.

9. The medium of claim 8 wherein the final loss layer directly notifies each side-output layer about the final objective of segmenting the skin lesion.

10. The medium of claim 8 , the method further comprising:
obtaining a training dermoscopic image and a training final score map;
convolving the training dermoscopic image in the plurality of convolutional layers;
deconvolving the outputs of the at least two convolutional layers of the plurality of convolutional layers;
obtaining trial side-output feature maps by applying loss functions to the deconvolved outputs of the at least two convolutional layers;
obtaining a trial concatenated feature map by concatenating the trial side-output feature maps with different weights;
convolving the trial concatenated feature map in the final convolutional layer followed by the loss layer to produce a trial final score map;
assessing variances of the trial final score map from the training final score map;
adjusting filters of the plurality of convolutional layers in response to the variances; and
repeating the preceding steps until the variances are less than a threshold variance vector.

11. The medium of claim 10 , the method further comprising adjusting deconvolution filters in response to the variances.

12. The medium of claim 8 wherein the plurality of convolutional layers comprise a VGG-16 neural network.

13. The medium of claim 12 wherein the at least two convolutional layers include conv2_2, conv3_3, conv4_3, and conv5_3 layers of the VGG-16 neural network.

14. The medium of claim 12 wherein the plurality of convolutional layers comprise a conv5_4 layer of dimensions 14×14×512, in addition to the layers of the VGG-16 neural network, and the conv5_4 layer is one of the at least two convolutional layers.

15. An apparatus comprising:
a memory in which computer executable instructions are stored; and
at least one processor, coupled to said memory, and operative by the computer executable instructions to perform a method comprising:
obtaining a dermoscopic image;
convolving the dermoscopic image in a plurality of convolutional layers;
obtaining deconvolved outputs of at least two convolutional layers of the plurality of convolutional layers;
obtaining side-output feature maps by applying loss functions to the deconvolved outputs of the at least two convolutional layers;
obtaining a first concatenated feature map by concatenating the side-output feature maps with different first weights;
obtaining a second concatenated feature map by concatenating the side-output feature maps with different second weights; and
producing a final score map by convolving the first and second concatenated feature maps in a final convolutional layer followed by a loss layer.

16. The apparatus of claim 15 , the method further comprising:
obtaining a training dermoscopic image and a training final score map;
convolving the training dermoscopic image in the plurality of convolutional layers;
deconvolving the outputs of the at least two convolutional layers of the plurality of convolutional layers;
obtaining trial side-output feature maps by applying loss functions to the deconvolved outputs of the at least two convolutional layers;
obtaining a trial concatenated feature map by concatenating the trial side-output feature maps with different weights;
convolving the trial concatenated feature map in the final convolutional layer followed by the loss layer to produce a trial final score map;
assessing variances of the trial final score map from the training final score map;
adjusting filters of the plurality of convolutional layers in response to the variances; and
repeating the preceding steps until the variances are less than a threshold variance vector.

17. The apparatus of claim 16 , the method further comprising adjusting deconvolution filters in response to the variances.

18. The apparatus of claim 15 wherein the plurality of convolutional layers comprise a VGG-16 neural network.

19. The apparatus of claim 18 wherein the at least two convolutional

Assignees

• IBM

Inventors

• Abedini Mani
• Bozorgtabar Seyedbehzad
• Chakravorty Rajib
• Demyanov Sergey
• Garnavi Rahil
• Ge Zongyuan

Classifications

• G06T2207/30088

  Skin; Dermal · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title