Retina image annotation, and related training methods and image processing models

Patent metadata

Field	Value
Publication number	US-12067726-B2
Application number	US-202218289547-A
Country	US
Kind code	B2
Filing date	May 5, 2022
Priority date	May 6, 2021
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

A computer-implemented method of annotating conventional retina images, the method comprising: receiving a conventional image of a retina, the conventional retina image captured using an image capture device; receiving an associated crosssectional image of said retina, the cross-sectional image captured using a crosssectional imaging system; determining a disease location in an image plane of the cross-sectional image; and generating annotation data for annotating the disease location in an image plane of the conventional image, by projecting the disease location from the image plane of the cross-sectional image into to the image plane of the conventional image, based on a known mapping between the cross-sectional image and the conventional image.

First claim

The invention claimed is:

1. A computer-implemented method of annotating conventional retina images, the method comprising:
receiving a conventional image of a retina, the conventional retina image captured using an image capture device;
receiving an associated cross-sectional image of said retina, the cross-sectional image captured using a cross-sectional imaging system;
determining a disease location in an image plane of the cross-sectional image; and
generating annotation data for annotating the disease location in an image plane of the conventional image, by projecting the disease location from the image plane of the cross-sectional image into to the image plane of the conventional image, based on a known mapping between the cross-sectional image and the conventional image.

2. The method of claim 1 , wherein the image plane of the cross-sectional image lies substantially perpendicular to the image plane of the conventional image, such that the cross sectional image maps to a scan line in the image plane of the conventional image, the disease location projected onto the scan line.

3. The method of claim 1 , wherein multiple cross-sectional images associated with the conventional retina image are received, and multiple disease locations are determined in respective image planes of the cross-sectional images and projected into the image plane of the conventional image to generate the annotation data.

4. The method of claim 3 , wherein the annotation data is generated via interpolation of the projected disease locations within the image plane of the conventional image.

5. The method of claim 4 , wherein the multiple cross-sectional images correspond to multiple scan lines in the image plane of the conventional image, and the annotation data is generated by interpolating the projected disease locations within one or more regions separating the scan lines.

6. The method of claim 1 , wherein the or each disease location is determined in the image plane of the cross-sectional image:
via automated image recognition applied to the cross-sectional image,
via manual annotation applied to the cross-sectional image, or
via a combination of automated image recognition and manual annotation.

7. The method of claim 1 , wherein the annotation data is generated in the form of a segmentation mask.

8. The method of claim 1 , wherein the segmentation mask assigns a severity level to each of at least some pixels of the conventional image, based on at least one of: a severity level(s) assigned to the disease location(s) in the cross-sectional image, and a depth of the disease location(s) within or behind the retina.

9. The method of claim 1 , wherein the conventional image is a fundus image, the image capture device being a fundus camera.

10. The method of claim 1 , wherein the cross sectional image is an optical coherence tomography (OCT) image, the cross-sectional imaging system being an OCT imaging system.

11. The method of claim 1 , multiple disease locations are determined in the image plane of the cross-sectional image, and a geometric voting algorithm is applied to the multiple disease locations to generate the annotation data based on at least one of: a severity level assigned to each disease location in the cross-sectional image, and a depth of each disease location within or behind the retina.

12. The method of claim 1 , applied to multiple conventional retina images and respective associated cross-sectional images, to generate respective annotation data for the multiple conventional retina images, wherein the multiple conventional retina images and their annotation data are used to train an image processing model to identify disease regions in conventional retina images, the multiple conventional retina images used as training inputs and their respective annotation data providing training ground truth, wherein the training inputs do not include the cross-sectional images.

13. The method of claim 12 , wherein the training inputs additionally comprise associated patient data.

14. An image processing system comprising:
an input configured to receive a conventional retina image; and
one or more processors configured to apply a trained image processing model to the conventional retina image, in order to identify at least one disease location therein, the image processing model trained in accordance with claim 12 .

15. A computer program embodying an image processing model, trained in accordance with claim 12 .

Assignees

• Vestel Elektronik Sanayi Ve Ticaret As

Inventors

• Aslan Muhammet
• Utine Canan Asli
• Topal Cihan
• Zengin Özlem Özden

Classifications

• G06V40/197

  Matching; Classification · CPC title

• G06V40/193

  Preprocessing; Feature extraction · CPC title

• G06V2201/03

  Recognition of patterns in medical or anatomical images · CPC title

• G16H20/00

  ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance · CPC title

• G16H50/30

  for calculating health indices; for individual health risk assessment · CPC title