Image processing apparatus, image processing method and computer-readable medium

Patent metadata

Field	Value
Publication number	US-12039704-B2
Application number	US-202117182402-A
Country	US
Kind code	B2
Filing date	Feb 23, 2021
Priority date	Sep 6, 2018
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

An image processing apparatus includes: an obtaining unit configured to obtain a first image of an eye to be examined; an image quality improving unit configured to generate a second image with at least one of lower noise and higher contrast than the obtained first image using the obtained first image as an input data of a learned model, wherein the learned model has been obtained by using training data including a second image with at least one of lower noise and higher contrast than a first image of an eye to be examined; and a display controlling unit configured to cause the obtained first image and the generated second image to be switched, juxtaposed or superimposed and displayed on a display unit.

First claim

What is claimed is:

1. An image processing apparatus, comprising at least one of (a) one or more processors connected to one or more memories storing a program including instructions executed by the one or more processors or (b) circuitry configured to function as:
an obtaining unit configured to obtain a plurality of first images of an eye to be examined obtained at different dates and times;
a generating unit configured to generate each of a plurality of second images with at least one of lower noise and higher contrast than each of the obtained plurality of first images using each of the obtained plurality of first images as input data of a learned model; and
a display controlling unit configured to control a display unit to change displays from one of (c) displays of the obtained plurality of first images and (d) displays of the generated plurality of second images to the other at a plurality of display areas in which a plurality of display areas of (c) and a plurality of display areas of (d) correspond to each other, according to a first instruction from an operator,
wherein the display controlling unit controls the display unit to change the displays from the displays of the generated plurality of second images corresponding to a first depth range in the eye to be examined to the displays of the generated plurality of second images corresponding to a second depth range that is at least partially different from the first depth range by changing a depth range from the first depth range to the second depth range according to a second instruction from an operator.

2. The image processing apparatus according to claim 1 , the at least one of (a) or (b) further configured to function as:
a selecting unit configured to select a learned model to be used by the generating unit from a plurality of learned models, based on an imaging condition of the obtained first image.

3. The image processing apparatus according to claim 1 ,
wherein the first image is a front image generated based on information in a range in a depth direction of an eye to be examined,
the at least one of (a) or (b) further configured to function as a selecting unit configured to selects a learned model corresponding to a range of a depth direction for generating the obtained first image as a learned model to be used by the generating unit from a plurality of learned models.

4. The image processing apparatus according to claim 1 , wherein:
the plurality of first images includes a plurality of front images generated based on information in a range in a depth direction of an eye to be examined; and
the display controlling unit simultaneously changes, upon the range in the depth direction for generating the front image being changed, the displays of the obtained plurality of first images or the displays of the generated plurality of second images to displays of a plurality of first images which are generated based on information in the changed range in the depth direction or displays of a plurality of second images that are generated from the generated plurality of first images.

5. The image processing apparatus according to claim 1 , wherein:
according to an instruction from an operator, the display controlling unit controls the display unit to simultaneously change displays from one of displays of a plurality of first analysis results corresponding to the obtained plurality of first images and displays of a plurality of second analysis results corresponding to the generated plurality of second images to the other.

6. The image processing apparatus according to claim 1 , the at least one of (a) or (b) further configured to function as a comparing unit configured to compare the obtained first image and the generated second image, and generate a color map image that is colored based on a comparison result,
wherein the display controlling unit causes the color map image to be displayed in a superimposed manner on the obtained first image or the generate second image on the display unit.

7. The image processing apparatus according to claim 1 , wherein:
the learned model is a learned model which has been obtained using training data including a fourth image with at least one of lower noise and higher contrast than a third image of an eye to be examined; and
a fourth image included in the training data of the learned model includes one of an image obtained by performing one kind of processing among averaging processing, maximum a posteriori processing, smoothing filter processing and gradation conversion processing, an image imaged with an imaging apparatus with higher performance than an imaging apparatus used for imaging of a third image of an eye to be examined, and an image obtained by an imaging process including a number of processes that is greater than a number of processes of an imaging process for imaging a third image of an eye to be examined.

8. An image processing method, comprising:
obtaining a plurality of first images of an eye to be examined obtained at different dates and times;
generating each of a plurality of second images with at least one of lower noise and higher contrast than each of the obtained plurality of first images using each of the obtained plurality of first images as input data of a learned model; and
controlling a display unit to change displays from one of (c) displays of the obtained plurality of first images and (d) displays of the generated plurality of second images to the other at a plurality of display areas in which a plurality of display areas of (c) and a plurality of display areas of (d) correspond to each other, according to a first instruction from an operator,
wherein the controlling includes controlling the display unit to change the displays from the displays of the generated plurality of second images corresponding to a first depth range in the eye to be examined to the displays of the generated plurality of second images corresponding to a second depth range that is at least partially different from the first depth range by changing a depth range from the first depth range to the second depth range according to a second instruction from an operator.

9. A non-transitory computer readable medium having stored thereon a program for causing, when executed by a processor, the processor to execute respective steps of the image processing method according to claim 8 .

10. The image processing apparatus according to claim 1 , wherein:
the learned model is a learned model which has been obtained using training data including a fourth image with at least one of lower noise and higher contrast than a third image of an eye to be examined.

11. The image processing apparatus according to claim 1 , wherein:
the plurality of first image includes a plurality of front images generated based on information in different ranges in a depth direction in an eye to be examined, which are obtained at the same dates and times.

Assignees

• Canon Kk

Inventors

• Uchida Hiroki
• Iwase Yoshihiko
• Sagano Osamu
• Tomita Ritsuya

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T5/90

  Dynamic range modification of images or parts thereof · CPC title

• G06N3/08

  Learning methods · CPC title