Marking and tracking an area of interest during endoscopy

Patent metadata

Field	Value
Publication number	US-9805469-B2
Application number	US-201514980880-A
Country	US
Kind code	B2
Filing date	Dec 28, 2015
Priority date	Sep 18, 2013
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

An area of interest of a patient's organ may be identified based on the presence of a possible lesion during an endoscopic procedure. The location of the area of interest may then be tracked relative to the camera view being displayed to the endoscopist in real-time or near real-time during the endoscopic procedure. If the area of interest is visually marked on the display, the visual marking is moved with the area of interest as it moves within the camera view. If the area of interest moves outside the camera view, a directional indicator may be displayed to indicate the location of the area of interest relative to the camera view to assist the endoscopist in relocating the area of interest.

First claim

What is claimed is:

1. One or more non-transitory computer storage media storing computer usable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
identifying an area of interest of a patient's organ within images captured during an endoscopic procedure;
providing a visual marker identifying the area of interest within displayed images of a camera view being viewed by an endoscopist;
tracking a location of the area of interest relative to the camera view during the endoscopic procedure;
determining the location of the area of interest has changed relative to the camera view;
determining if the area of interest remains within the camera view;
if the area of interest remains within the camera view, moving the visual marker based on the change in the location of the area of interest; and
if the area of interest has moved outside of the camera view, providing a visual directional marker based on the location of the area of interest relative to the camera view.

2. The one or more computer storage media of claim 1 , wherein the area of interest is manually identified by the endoscopist.

3. The one or more computer storage media of claim 2 , wherein the area of interest is manually identified by the endoscopist by:
receiving input from the endoscopist via an input device marking an area within the displayed images as the area of interest.

4. The one or more computer storage media of claim 1 , wherein the area of interest is automatically identified.

5. The one or more computer storage media of claim 4 , wherein the area of interest is automatically identified by:
providing a library of lesion shapes;
performing edge detection to determine a shape of an area within the patient's organ;
comparing the shape with the library of lesion shapes;
determining the shape as matching a lesion shape from the library of lesion shapes; and
identifying the area as an area of interest based on the shape matching the lesion shape from the library of lesion shapes.

6. The one or more computer storage media of claim 4 , wherein the area of interest is automatically identified by:
presenting the images in white light to the endoscopist;
analyzing the images using modified color conditions as a background process; and
identifying the area of interest based on analysis of the images using the modified color conditions.

7. The one or more computer storage media of claim 6 , wherein the operations further comprise: changing the images displayed to the endoscopist to modified color images based on a modified color channel setting used to identify the area of interest.

8. The one or more computer storage media of claim 1 , wherein the visual marker comprises at least one selected from the following: a box surrounding the area of interest, highlighting an edge of the area of interest, coloring the area of interest, and dimming areas outside of the area of interest.

9. The one or more computer storage media of claim 1 , wherein tracking the location of the area of interest relative to a camera view during the endoscopic procedure comprises employing computer vision techniques to analyze images captured during the endoscopic procedure to determine movement of the area of interest relative to the camera view.

10. A method in a clinical computing environment for tracking an area of interest of a patient's organ during an endoscopic procedure, the method comprising:
identifying the area of interest within a camera view during the endoscopic procedure;
determining the area of interest has moved outside the camera view during the endoscopic procedure; and
displaying a directional indication on displayed images identifying a location of the area of interest relative to the camera view during the endoscopic procedure.

11. The method of claim 10 , wherein identifying the area of interest comprises receiving input from an endoscopist via an input device marking an area within the displayed images as the area of interest.

12. The method of claim 10 , wherein the area of interest is automatically identified by:
providing a library of lesion shapes;
performing edge detection to determine a shape of an area within the patient's organ;
comparing the shape with the library of lesion shapes;
determining the shape as matching a lesion shape from the library of lesion shapes; and
identifying the area as an area of interest based on the shape matching the lesion shape from the library of lesion shapes.

13. The method of claim 10 , wherein the area of interest is automatically identified by:
presenting the displayed images in white light to an endoscopist;
analyzing captured images using modified color conditions as a background process; and
identifying the area of interest based on analysis of the captured images using the modified color conditions.

14. The method of claim 13 , wherein the method further comprises: changing the displayed images to modified color images based on a modified color channel setting used to identify the area of interest.

15. The method of claim 10 , wherein the method further comprises displaying a visual marker identifying the area of interest on the displayed images when the area of interest is within the displayed images, the visual marker comprising at least one selected from the following: a box surrounding the area of interest, highlighting an edge of the area of interest, coloring the area of interest, and dimming areas outside of the area of interest.

16. The method of claim 10 , wherein determining the area of interest has moved outside the camera view during the endoscopic procedure comprises employing computer vision techniques to analyze images captured during the endoscopic procedure to determine movement of the area of interest relative to the camera view.

17. A system in a clinical computing environment for tracking an area of interest of a patient's organ during an endoscopic procedure, the system comprising:
one or more processors; and
one or more non-transitory computer storage media storing instructions to cause the one or more processors to:
identify the area of interest within a camera view during the endoscopic procedure,
determine the area of interest has moved outside the camera view during the endoscopic procedure, and
display a directional indication on displayed images identifying a location of the area of interest relative to the camera view during the endoscopic procedure.

18. The system of claim 17 , wherein the area of interest is manually identified by an endoscopist.

19. The system of claim 17 , wherein the area of interest is automatically identified by the system analyzing images captured during the endoscopic procedure.

20. The system of claim 17 , wherein determining the area of interest has moved outside the camera view during the endoscopic procedure comprises employing computer vision techniques to analyze images captured during the endoscopic procedure to determine movement of the area of interest relative to the camera view.

Assignees

• Cerner Innovation Inc

Inventors

• Staples Ii Alan Harris
• Ramsey Karen Kaye
• Hunt Bryan Michael

Classifications

• A61B1/0004

  for electronic operation · CPC title

• A61B1/000095 Primary

  for image enhancement · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title

• G06T7/248

  involving reference images or patches · CPC title