Automatically detecting eye type in retinal fundus images

Patent metadata

Field	Value
Publication number	US-10405739-B2
Application number	US-201615047734-A
Country	US
Kind code	B2
Filing date	Feb 19, 2016
Priority date	Oct 23, 2015
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

A computer-implemented method includes obtaining an image of a retinal fundus. A plurality of features is extracted from the image of the retinal fundus. The plurality of features includes at least one feature based on anatomical domain knowledge of the retinal fundus and at least one response of a pre-trained deep convolutional neural network to at least a portion of the image of the retinal fundus. The retinal fundus is determined to belong to a left eye or a right eye, based on an analysis of the plurality of features.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
obtaining an image of a retinal fundus;
extracting a supervised feature of the retinal fundus based on anatomical domain knowledge of the retinal fundus from the image of the retinal fundus;
deriving an unsupervised feature of the retinal fundus from a response of a pre-trained deep convolutional neural network to at least a portion of the image of the retinal fundus, wherein the deriving comprises:
passing an image of a region of interest that has been cropped from the image of the retinal fundus through a stack of N learned convolutional layers of the pre-trained deep convolutional neural network, wherein N is a whole number between one and sixteen;
applying principal component analysis on responses of an (N−1)th layer of the N learned convolutional layers while discarding a response of an Nth layer of the N learned convolutional layers, wherein the Nth layer is trained for one thousand classes of a training dataset, and the (N−1)th layer generates more than four thousand responses as potential unsupervised features; and
extracting the unsupervised feature from a set of principal components obtained by the applying; and

determining whether the retinal fundus belongs to a left eye or a right eye, based on an analysis of a combination of the supervised feature and the unsupervised feature.

2. The computer-implemented method of claim 1 , wherein the supervised feature comprises a cross-sectional vote-based probability score for the image of the retinal fundus.

3. The computer-implemented method of claim 2 , wherein the extracting the supervised feature comprises:
cropping a region of interest from the image of the retinal fundus;
defining a search region within the region of interest;
defining a plurality of horizontal cross sections with the search region;
generating a plurality of votes for the plurality of horizontal cross sections, wherein a given vote of the plurality of votes indicates whether a corresponding cross section of the plurality of horizontal cross sections likely depicts the left eye or the right eye; and
computing a probability score for the image of the retinal fundus, based on the plurality of votes.

4. The computer-implemented method of claim 3 , wherein the region of interest is centered on an optic disc depicted in the image of the retinal fundus.

5. The computer-implemented method of claim 4 , wherein the search region extends a predefined number above pixels above and below a center of the optic disc.

6. The computer-implemented method of claim 3 , wherein the generating comprises, for a given cross section of the plurality of horizontal cross sections:
smoothing a pixel intensity distribution along the given cross section;
computing a peak point of the pixel intensity distribution, subsequent to the smoothing;
voting for the left eye when the peak point is located to the right of a center of an optic disc depicted in the image of the retinal fundus; and
voting for the right eye when the peak point is located to the left of the center of the optic disc.

7. The computer-implemented method of claim 6 , wherein the center of the optic disc is identified by performing operations comprising:
identifying a start point of the optic disc in the given cross section as a highest point along a gradient of pixel intensity values in the image of the retinal fundus;
identifying an end point of the optic disc in the given cross section as a lowest point along the gradient of pixel intensity values; and
designating a middle point between the start point and the end point as the center of the optic disc for the given cross section.

8. The computer-implemented method of claim 1 , wherein the supervised feature comprises relative blood vessel densities on the left side and the right side of an optic disc depicted in the image of the retinal fundus.

9. The computer-implemented method of claim 8 , wherein the extracting the supervised feature comprises:
dividing a blood vessel mask image corresponding to the image of the retinal fundus into two parts, based on a location of a center of the optic disc, where a first part of the two parts depicts the left side of the optic disc, and a second part of the two parts depicts the right side of the optic disc; and
computing the relative blood vessel densities for the two parts.

10. The computer-implemented method of claim 1 , wherein the supervised feature comprises an orientation of a major retinal blood vessel with respect to an optic disc depicted in the image of the retinal fundus.

11. The computer-implemented method of claim 10 , wherein the extracting the supervised feature comprises:
applying an optic disc-centered mask to a binary blood vessel mask corresponding to the image of the retinal fundus;
detecting individual blood vessels in the binary blood vessel mask that reside within a region inside the optic disc-centered mask;
computing an area and a centroid for each blood vessel of the individual blood vessels;
deriving the orientation from an angle between a center of the optic disc and the centroid of the blood vessel of the individual blood vessels for which the area is largest.

12. The computer-implemented method of claim 11 , wherein the optic-disc centered mask is obtained by performing operations comprising:
drawing a circle of a predefined radius around the optic disc in the image of the retinal fundus, wherein the circle defines the optic disc-centered mask.

13. The computer-implemented method of claim 1 , wherein the determining is performed using a support vector machine that has been trained using a plurality of features including the supervised feature and the unsupervised feature.

14. The computer-implemented method of claim 1 , wherein the supervised feature is extracted from an image of retinal blood vessels that has been segmented from the image of the retinal fundus.

15. The computer-implemented method of claim 4 , further comprising locating a position of the optic disc in the image of the retinal fundus by:
generating a green channel retinal image from the image of the retinal fundus;
applying a plurality of non-overlapping sliding windows on the green channel retinal image and on a binary blood vessel mask corresponding to the green channel retinal image; and
selecting a location of a sliding window of the plurality of sliding windows that has a maximum blood vessel density and a maximum intensity value as the position of the optic disc.

16. The computer-implemented method of claim 4 , wherein an index of a starting cross section of the plurality of horizontal cross sections is R c−n , an index of an ending cross section of the plurality of horizontal cross sections is R c+n , c is an index of a cross section of the plurality of horizontal cross sections that passes through a center of the optic disc, n=r*a, a=0.65, and n is smaller than r.

17. The computer-implemented method of claim 6 , wherein the cross-sectional vote-based probability score is computed by:
repeating the smoothing, the computing the peak point, the voting for the left eye, and the voting for the right eye for every cross section of the plurality of horizontal cross sections, to generate a plurality of votes; and
computing the cross-sectional vote-based probability score for the left eye as a percentage of the plurality of votes that favors the left eye; and
computing the cross-sectional vote-based probability score for the right eye as a percentage of the plurality of votes that favors the right eye.

1

Assignees

• IBM

Inventors

• Chakravorty Rajib
• Garnavi Rahil
• Roy Pallab

Classifications

• A61B3/12 Primary

  for looking at the eye fundus, e.g. ophthalmoscopes (A61B3/13 takes precedence) · CPC title

• A61B3/0025 Primary

  characterised by electronic signal processing, e.g. eye models · CPC title

• G06V40/197

  Matching; Classification · CPC title

• G06V40/193

  Preprocessing; Feature extraction · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title