Classification of severity of pathological condition using hybrid image representation

Patent metadata

Field	Value
Publication number	US-10169872-B2
Application number	US-201715426634-A
Country	US
Kind code	B2
Filing date	Feb 7, 2017
Priority date	Nov 2, 2016
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

A computer-implemented method obtains at least one image from which severity of a given pathological condition presented in the at least one image is to be classified. The method generates a hybrid image representation of the at least one obtained image. The hybrid image representation comprises a concatenation of a discriminative pathology histogram, a generative pathology histogram, and a fully connected representation of a trained baseline convolutional neural network. The hybrid image representation is used to train a classifier to classify the severity of the given pathological condition presented in the at least one image. One non-limiting example of a pathological condition whose severity can be classified with the above method is diabetic retinopathy.

First claim

What is claimed is:

1. A method comprising:
obtaining, at a digital image processor, at least one image from which severity of a given pathological condition presented in the at least one image is to be classified; and
generating, at the digital image processor, a hybrid image representation of the at least one obtained image, wherein the hybrid image representation comprises a concatenation of a discriminative pathology histogram, a generative pathology histogram, and a fully connected representation of a trained baseline convolutional neural network, and further wherein the discriminative pathology histogram and the generative pathology histogram are each computed using a word occurrence count vector modeling process;
training a classifier using the hybrid image representation;
classifying the severity of the given pathological condition presented in the at least one image based on the trained classifier; and
wherein the steps of the method are performed by a computing device comprising a processor coupled to a memory.

2. The method of claim 1 , wherein the given pathological condition presented in the at least one obtained image comprises diabetic retinopathy.

3. The method of claim 2 , wherein the classifier classifies the diabetic retinopathy in the at least one obtained image into one of a plurality of diabetic retinopathy severity classifications.

4. The method of claim 1 , wherein the classifier comprises a random forest classifier.

5. The method of claim 1 wherein the discriminative pathology histogram is computed utilizing a random forest dictionary and a linear support vector machine classifier and the generative pathology histogram is computed utilizing a Gaussian Mixture Model dictionary and at least one Fisher vector.

6. The method of claim 1 , wherein generating the hybrid image representation further comprises extracting a set of patches from the at least one obtained image.

7. The method of claim 6 , wherein generating the hybrid image representation further comprises passing the extracted set of patches through the trained baseline convolutional neural network to obtain a fully connected representation of the extracted set of patches in the form of a set of feature descriptors for each patch.

8. The method of claim 7 , wherein generating the hybrid image representation further comprises encoding the set of feature descriptors for each patch using the classifier.

9. The method of claim 8 , wherein the set of feature descriptors for each patch is encoded using the word occurrence count vector modeling process.

10. The method of claim 8 , wherein generating the hybrid image representation further comprises clustering the encoded feature descriptors in each patch.

11. The method of claim 10 , wherein generating the hybrid image representation further comprises computing the discriminative pathology histogram from the clustered feature descriptors.

12. An apparatus, comprising:
at least one processor; and
a memory operatively coupled to the processor and configured to:
obtain, at a digital image processor, at least one image from which severity of a given pathological condition presented in the at least one image is to be classified;
generate, at the digital image processor, a hybrid image representation of the at least one obtained image, wherein the hybrid image representation comprises a concatenation of a discriminative pathology histogram, a generative pathology histogram, and a fully connected representation of a trained baseline convolutional neural network, and further wherein the discriminative pathology histogram and the generative pathology histogram are each computed using a word occurrence count vector modeling process
train a classifier using the hybrid image representation; and
classify the severity of the given pathological condition presented in the at least one image based on the trained classifier.

13. A computer program product comprising a processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by the one or more processors implement steps of:
obtaining, at a digital image processor, at least one image from which severity of a given pathological condition presented in the at least one image is to be classified;
generating, at the digital image processor, a hybrid image representation of the at least one obtained image, wherein the hybrid image representation comprises a concatenation of a discriminative pathology histogram, a generative pathology histogram, and a fully connected representation of a trained baseline convolutional neural network, and further wherein the discriminative pathology histogram and the generative pathology histogram are each computed using a word occurrence count vector modeling process;
training a classifier using the hybrid image representation; and
classifying the severity of the given pathological condition presented in the at least one image based on the trained classifier.

14. The computer program product of claim 13 , wherein the given pathological condition presented in the at least one obtained image comprises diabetic retinopathy, and wherein the classifier classifies the diabetic retinopathy in the at least one obtained image into one of a plurality of diabetic retinopathy severity classifications.

15. The computer program product of claim 13 , wherein the classifier comprises a random forest classifier.

16. The computer program product of claim 13 , wherein the discriminative pathology histogram is computed utilizing a random forest dictionary and a linear support vector machine classifier and the generative pathology histogram is computed utilizing a Gaussian Mixture Model dictionary and at least one Fisher vector.

17. The computer program product of claim 13 , wherein generating the hybrid image representation further comprises:
extracting a set of patches from the at least one obtained image;
passing the extracted set of patches through the trained baseline convolutional neural network to obtain a fully connected representation of the extracted set of patches in the form of a set of feature descriptors for each patch; and
encoding the set of feature descriptors for each patch using the classifier.

18. The computer program product of claim 17 , wherein the set of feature descriptors for each patch is encoded using the word occurrence count vector modeling process.

19. The computer program product of claim 17 , wherein generating the hybrid image representation further comprises clustering the encoded feature descriptors in each patch.

20. The computer program product of claim 19 , wherein generating the hybrid image representation further comprises computing the discriminative pathology histogram from the clustered feature descriptors.

Assignees

• IBM

Inventors

• Garnavi Rahil
• Mahapatra Dwarikanath
• Roy Pallab
• Sedai Suman
• Tennakoon Ruwan B

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/809

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/24143

  Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN] · CPC title

• G06F18/254

  of classification results, e.g. of results related to same input data · CPC title