Image processing apparatus, image processing method and computer-readable medium

Patent metadata

Field	Value
Publication number	US-12094082-B2
Application number	US-202117343207-A
Country	US
Kind code	B2
Filing date	Jun 9, 2021
Priority date	Dec 26, 2018
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

An image processing apparatus includes: an obtaining unit configured to obtain a first medical image of an object under examination; an image quality improving unit configured to generate, from the obtained first medical image, a second medical image with image quality higher than image quality of the obtained first medical image using a learned model; a comparing unit configured to compare an analysis result obtained by analyzing the obtained first medical image and an analysis result obtained by analyzing the generated second medical image; and a display controlling unit configured to cause a comparison result obtained by the comparing unit to be displayed on a display unit.

First claim

What is claimed is:

1. An image processing apparatus, comprising at least one of (a) one or more processors connected to one or more memories storing a program including instructions executed by the one or more processors and (b) circuitry configured to function as:
an obtaining unit configured to obtain a first medical image of an object under examination;
an image quality improving unit configured to generate, from the obtained first medical image, a second medical image with image quality higher than image quality of the obtained first medical image using a learned model;
a comparing unit configured to compare an analysis result obtained by analyzing the obtained first medical image and an analysis result obtained by analyzing the generated second medical image; and
a display controlling unit configured to cause a comparison result obtained by the comparing unit to be displayed on a display unit, wherein:
the first medical image is a motion contrast en-face image in a range in a depth direction of an eye under examination; and
the analysis result is at least one of a value relating to a blood vessel and a value relating to an avascular zone.

2. The image processing apparatus according to claim 1 , wherein:
the comparing unit calculates a difference between the analysis result for the obtained first medical image and the analysis result for the generated second medical image, and generates a color map image that is colored based on the difference; and
the display controlling unit causes the color map image to be displayed on the display unit as the comparison result.

3. The image processing apparatus according to claim 2 , wherein:
the display controlling unit causes the color map image to be displayed on the display unit in a superimposed manner on the obtained first medical image or the generated second medical image.

4. The image processing apparatus according to claim 1 , wherein:
the comparing unit calculates a difference between the analysis result for the obtained first medical image and the analysis result for the generated second medical image; and
the display controlling unit causes a warning to be displayed on the display unit as the comparison result if the difference is greater than a predetermined value or a number of pixels for which the difference is greater than a predetermined value is greater than another predetermined value.

5. The image processing apparatus according to claim 1 , wherein:
the comparing unit calculates a difference between the analysis result for the obtained first medical image and the analysis result for the generated second medical image; and
the display controlling unit causes a comparison result to be displayed on the display unit by causing a region in which the difference is greater than a predetermined value to be displayed on the display unit distinguishably from another region in which the difference is equal to or less than the predetermined value.

6. The image processing apparatus according to claim 1 , wherein:
the obtaining unit obtains a plurality of first medical images that are en-face images generated based on information pertaining to a common range in a depth direction of an object under examination;
the image quality improving unit generates, from the obtained plurality of first medical images, a plurality of second medical images with image quality higher than image quality of the obtained plurality of first medical images using the learned model; and
the comparing unit compares the obtained plurality of first medical images and the generated plurality of second medical images before and after the image quality improving processing.

7. The image processing apparatus according to claim 1 , wherein training data of the learned model includes at least one of:
an image obtained by performing at least one kind of processing among averaging processing, maximum a posteriori processing, smoothing filter processing and gradation conversion processing;
an image imaged with an imaging apparatus with higher performance than an imaging apparatus used for imaging of a first medical image of an object under examination; and
an image obtained by an imaging process including a number of steps that is greater than a number of steps of an imaging process for imaging a first medical image of an object under examination.

8. The image processing apparatus according to claim 1 , wherein the display controlling unit causes at least one of:
an analysis result relating to the generated second medical image that is an analysis result generated using a learned model for generating analysis results that is obtained by using medical images of an object under examination, to be displayed on the display unit;
an object detection result relating to the generated second medical image that is an object detection result generated using a learned model for object recognition that is obtained by using medical images of an object under examination;
a segmentation result relating to the generated second medical image that is a segmentation result generated using a learned model for segmentation that is obtained by using medical images of an object under examination; and
a similar case image relating to the generated second medical image that is a similar case image searched for by using a learned model for similar case image searching that is obtained by using medical images of an object under examination, to be displayed on the display unit.

9. The image processing apparatus according to claim 1 , wherein:
the display controlling unit causes information relating to a difference between an image generated using a generative adversarial network or an auto-encoder into which the generated second medical image is input, and the generated second medical image input into the generative adversarial network or the auto-encoder to be displayed on the display unit as information relating to an abnormal site.

10. The image processing apparatus according to claim 1 , wherein:
an instruction from an examiner relating to a processing by the image quality improving unit or an instruction from an examiner relating to control of a display based on the comparison result is information obtained using at least one learned model among a learned model for character recognition, a learned model for speech recognition and a learned model for gesture recognition.

11. The image processing apparatus according to claim 1 , wherein:
a file name of the generated second medical image includes information indicating that the generated second medical image is an image generated by performing the image quality improving processing, in a state in which the information can be edited according to an instruction from an examiner.

12. An image processing method, comprising:
obtaining a first medical image of an object under examination;
generating, from the obtained first medical image, a second medical image with image quality higher than image quality of the obtained first medical image using a learned model;
comparing an analysis result obtained by analyzing the obtained first medical image and an analysis result obtained by analyzing the generated second medical image; and
causing a comparison result obtained by the comparing to be displayed on a display unit, wherein:
the first medical image is a motion contrast en-face image in a range in a depth direction of an eye under examination; and
the analysis result is at least one of a value relating to a blood vessel and a value relating to an avascular zone.

13. A non-transitory computer-readable medium having stored thereon a program that, upon being executed by a processor, caus

Assignees

• Canon Kk

Inventors

• Makihira Tomoyuki
• Uchida Hiroki
• Tomita Ritsuya

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T12/30

  Image post-processing, e.g. metal artefact correction · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/0475

  Generative networks · CPC title