Endoscope system, endoscope image processing method, and storage medium

Patent metadata

Field	Value
Publication number	US-11642005-B2
Application number	US-202117329446-A
Country	US
Kind code	B2
Filing date	May 25, 2021
Priority date	Nov 28, 2018
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

A processor of an endoscope system detects from the observation images a lesion area which is an observation target of the endoscope, judges a level of an oversight risk which is a risk that an operator may overlook the lesion area, on the basis of the observation images, and controls notification methods of detection of the lesion area on the basis of the level of the oversight risk. The processor displays the marker image indicating the lesion area only in a second image region of the display apparatus when a level of the oversight risk relating to the lesion area is a first level and displays the marker image in both a first image region and the second image region which is smaller than the first image region when a level of the oversight risk relating to the lesion area is a second level higher than the first level.

First claim

What is claimed is:

1. An endoscope system comprising:
a processor including hardware; and
a display apparatus,
the processor being configured to:
sequentially input a plurality of observation images obtained by picking up images of an object with an endoscope,
detect from the observation images a lesion area which is an observation target of the endoscope,
judge a level of an oversight risk which is a risk that an operator overlooks the lesion area, on a basis of the observation images, and
control notification methods of detection of the lesion area on a basis of the level of the oversight risk,
the display apparatus including a first image region and a second image region that are for displaying a marker image indicating the lesion area, the second image region being smaller than the first image region, to notify the operator of detection of the lesion area on a basis of control of the notification methods,
the processor causing the marker image to be displayed only in the second image region in a case where a level of the oversight risk relating to the lesion area is a first level and causing the marker image to be displayed in both the first image region and the second image region in a case where a level of the oversight risk relating to the lesion area is a second level higher than the first level.

2. The endoscope system according to claim 1 ,
wherein the processor analyzes the oversight risk on a basis of a state of the lesion area.

3. The endoscope system according to claim 2 ,
wherein the processor estimates a size of the lesion area.

4. The endoscope system according to claim 2 ,
wherein the processor analyzes a position of the lesion area in the observation images.

5. The endoscope system according to claim 2 ,
wherein the processor includes a lesion density analysis unit configured to analyze density of the lesion area.

6. The endoscope system according to claim 2 ,
wherein the processor analyzes a shape of the lesion area.

7. The endoscope system according to claim 1 ,
wherein the processor analyzes the oversight risk on a basis of a state of the observation images.

8. The endoscope system according to claim 7 ,
wherein the processor analyzes an exposure state of the observation images.

9. The endoscope system according to claim 7 ,
wherein the processor analyzes a degree of focusing of a focus of the observation images.

10. The endoscope system according to claim 7 ,
wherein the processor analyzes whether or not a residue or bleeding exists in the observation images.

11. The endoscope system according to claim 7 ,
wherein the processor analyzes whether or not the observation images are opaque or dirty.

12. The endoscope system according to claim 1 ,
wherein the processor analyzes the oversight risk on a basis of a movement state of the lesion area in the observation images.

13. The endoscope system according to claim 12 ,
wherein the processor analyzes change of movement speed of the lesion area in the observation images.

14. The endoscope system according to claim 12 ,
wherein the processor analyzes change of a position of the lesion area in the observation images.

15. The endoscope system according to claim 1 ,
wherein the processor performs notification control to generate a marker image indicating the lesion area and superimpose the marker image on the observation images and makes at least one of color, a thickness or a size of the marker image different in accordance with a level of the oversight risk of the lesion area.

16. The endoscope system according to claim 1 ,
wherein the display apparatus is provided in plurality, and each of the display apparatuses makes notification of detection of the lesion area using notification means different from each other.

17. The endoscope system according to claim 16 ,
wherein the processor selects a display apparatus which becomes a notification target from the display apparatus in plurality and controls the notification methods for the selected display apparatus.

18. The endoscope system according to claim 16 ,
wherein the notification means is one of an image, voice or vibration.

19. The endoscope system according to claim 17 ,
wherein the notification method setting unit determines the notification methods for the respective notification means in accordance with a combination of the notification means in plurality selected by a notification means selection unit.

20. An endoscope image processing method comprising:
sequentially inputting a plurality of observation images obtained by picking up images of an object with an endoscope;
detecting from the observation images a lesion area which is an observation target of the endoscope;
judging a level of an oversight risk which is a risk that an operator overlooks the lesion area, on a basis of the observation images;
controlling notification methods of detection of the lesion area on a basis of the level of the oversight risk;
notifying the operator of detection of the lesion area on a basis of control of the notification methods using a display apparatus including a first image region and a second image region that are for displaying a marker image indicating the lesion area, the second image region being smaller than the first image region; and
causing the marker image to be displayed only in the second image region in a case where a level of the oversight risk relating to the lesion area is a first level and causing the marker image to be displayed in both the first image region and the second image region in a case where a level of the oversight risk relating to the lesion area is a second level higher than the first level.

21. A computer-readable storage medium which is a non-transitory computer-readable recording medium storing a program executed by a computer, the computer-readable storage medium storing an endoscope image processing program for causing the computer to execute:
sequentially acquiring a plurality of observation images obtained by picking up images of an object with an endoscope;
detecting from the observation images a lesion area which is an observation target of the endoscope;
judging a level of an oversight risk which is a risk that an operator overlooks the lesion area, on a basis of the observation images;
controlling notification methods of detection of the lesion area on a basis of the level of the oversight risk;
notifying the operator of detection of the lesion area on a basis of control of the notification methods using a display apparatus including a first image region and a second image region that are for displaying a marker image indicating the lesion area, the second image region being smaller than the first image region; and
displaying the marker image only in the second image region in a case where a level of the oversight risk relating to the lesion area is a first level and displaying the marker image in both the first image region and the second image region in a case where a level of the oversight risk relating to the lesion area is a second level higher than the first level.

Assignees

• Olympus Corp

Inventors

• Kubota Akihiro
• Taniguchi Katsuyoshi
• Kanda Yamato
• Kimura Mitsutaka

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title

• A61B1/045

  Control thereof · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G06T2207/30168

  Image quality inspection · CPC title