Early prediction of age related macular degeneration by image reconstruction

Patent metadata

Field	Value
Publication number	US-10098533-B2
Application number	US-201715854984-A
Country	US
Kind code	B2
Filing date	Dec 27, 2017
Priority date	Sep 23, 2016
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

An AMD prediction model utilizes an OCT image estimation model. The OCT image estimation module is created by segmenting an OCT image to generate an OCT projection image for each of multiple biological layers; extracting from each of the generated OCT projection images a first set of features; extracting a second set of features from an input retinal fundus image; for each respective biological layer, registering the input retinal fundus image to the respective OCT projection image by matching at least some of the second set of features with corresponding ones of the first set of features; repeating the above with changes to the input retinal fundus image; and modelling how the changes to the input retinal fundus image are manifest at the correspondingly registered projection images. Estimated OCT projection images can then be generated for the multiple biological layers from a given retinal fundus image.

First claim

What is claimed is:

1. A method comprising:
a) segmenting an optical coherence tomography (OCT) image to generate an OCT projection image for each of multiple biological layers;
b) extracting from each of the generated OCT projection images a first set of features;
c) extracting a second set of features from an input retinal fundus image;
d) for each respective biological layer, registering the input retinal fundus image to the respective OCT projection image by matching at least some of the second set of features with corresponding ones of the first set of features;
e) repeating elements a) through d) with changes to the input retinal fundus image;
f) modelling how the changes to the input retinal fundus image are manifest at the correspondingly registered projection images;
g) storing in a computer readable memory a program that utilizes the modelling to generate estimated OCT projection images for the multiple biological layers from a given retinal fundus image;
h) using the stored program to generate estimated OCT projection images for the multiple biological layers from a patient's retinal fundus image;
i) extracting from each of the estimated OCT projection images a first set of features;
j) extracting a second set of features from the patient's retinal fundus image;
k) evaluating the first set of features extracted from each of the estimated OCT projection images in combination with the second set of features extracted from the patient's retinal fundus image to predict age related macular degeneration (AMD) in the patient; and
l) outputting an indication of the predicted AMD in the patient for use in diagnosing AMD in the patient.

2. The method according to claim 1 , wherein the stored program is used to predict AMD in the patient in the absence of inputting to the program any OCT images of the patient.

3. The method according to claim 1 , wherein each of the estimated OCT projection images are generated by an auto-encoder.

4. The method according to claim 3 , wherein there are a plurality of convolution auto-encoders in a stacked arrangement, each of the auto-encoders configured to generate from each of the input retinal fundus images the estimated OCT projection image for only one of the biological layers.

5. The method according to claim 1 , wherein each of the first and second sets of features comprise vessel based landmark points.

6. The method according to claim 1 , wherein at least some of the changes to the input retinal fundus image are manifest at the correspondingly registered projection images as changes to size, number and/or location of drusens.

7. A computer readable memory storing an executable program comprising:
a) code to segment an optical coherence tomography (OCT) image to generate an OCT projection image for each of multiple biological layers;
b) code to extract from each of the generated OCT projection images a first set of features;
c) code to extract a second set of features from an input retinal fundus image;
d) code to register the input retinal fundus image to the respective OCT projection images of each respective biological layer by matching at least some of the second set of features with corresponding ones of the first set of features;
e) code to repeat elements a) through d) with changes to the input retinal fundus image;
f) code to model how the changes to the input retinal fundus image are manifest at the correspondingly registered projection images;
g) code to utilize the model for generating estimated OCT projection images for the multiple biological layers from a given retinal fundus image
h) code to use the stored program to generate estimated OCT projection images for the multiple biological layers from a patient's retinal fundus image;
i) code to extract from each of the estimated OCT projection images a first set of features;
j) code to extract a second set of features from the patient's retinal fundus image;
k) code to evaluate the first set of features extracted from each of the estimated OCT projection images in combination with the second set of features extracted from the patient's retinal fundus image to predict age related macular degeneration (AMD) in the patient; and
l) code to output an indication of the predicted AMD in the patient for use in diagnosing AMD in the patient.

8. The computer readable memory according to claim 7 , wherein the code to evaluate and to predict AMD operates in the absence of any actual OCT images being input to the executable program after the changes are modelled.

9. The computer readable memory according to claim 7 , wherein the model comprises an auto-encoder.

10. The computer readable memory according to claim 9 , wherein the model comprises a plurality of convolution auto-encoders in a stacked arrangement, each of the auto-encoders configured to generate from each of the input retinal fundus images the estimated OCT projection image for only one of the biological layers.

11. The computer readable memory according to claim 7 , wherein each of the first and second sets of features comprise vessel based landmark points.

12. The computer readable memory according to claim 7 , wherein at least some of the changes to the input retinal fundus image are manifest at the correspondingly registered projection images as changes to size, number and/or location of drusen.

13. An apparatus comprising:
one or more memories comprising computer-readable code and one or more processors, wherein the one or more processors are configured, in response to execution of the computer-readable code, to cause the apparatus to perform actions comprising:
a) segmenting an optical coherence tomography (OCT) image to generate an OCT projection image for each of multiple biological layers;
b) extracting from each of the generated OCT projection images a first set of features;
c) extracting a second set of features from an input retinal fundus image;
d) for each respective biological layer, registering the input retinal fundus image to the respective OCT projection image by matching at least some of the second set of features with corresponding ones of the first set of features;
e) repeating elements a) through d) with changes to the input retinal fundus image;
f) modelling how the changes to the input retinal fundus image are manifest at the correspondingly registered projection images;
g) storing in the one or more memories a program that utilizes the modelling to generate estimated OCT projection images for the multiple biological layers from a given retinal fundus image;
h) using the stored program to generate estimated OCT projection images for the multiple biological layers from a patient's retinal fundus image;
i) extracting from each of the estimated OCT projection images a first set of features;
j) extracting a second set of features from the patient's retinal fundus image;
k) evaluating the first set of features extracted from each of the estimated OCT projection images in combination with the second set of features extracted from the patient's retinal fundus image to predict age related macular degeneration (AMD) in the patient; and
l) outputting an indication of the predicted AMD in the patient for use in diagnosing AMD in the patient.

14. The apparatus according to claim 13 , wherein the stored program is used to predict AMD in the patient in the absence of inputting to the program any OCT images of the patient.

15. The apparatus according to claim 13 , wherein each of the estimated OCT projection images are generated by an auto-encoder.

16. The apparatus according to claim 15 , whe

Assignees

• IBM

Inventors

• Chakravorty Rajib
• Garnavi Rahil
• Mahapatra Dwarikanath
• Roy Pallab
• Sedai Suman

Classifications

• A61B3/102 Primary

  for optical coherence tomography [OCT] · CPC title

• A61B3/0025 Primary

  characterised by electronic signal processing, e.g. eye models · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06T12/30

  Image post-processing, e.g. metal artefact correction · CPC title

• G06T7/10 Primary

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title