Frame processing of imaging scope data for user interface presentation

Patent metadata

Field	Value
Publication number	US-12396621-B2
Application number	US-202117644322-A
Country	US
Kind code	B2
Filing date	Dec 14, 2021
Priority date	Dec 14, 2021
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

Improved medical scope devices and systems are provided to with two imaging modes and user interface features based on the imaging modes. A medical scope has a shaft with a light emitter at the distal tip providing illumination light and an optical assembly including a wide-angle lens element. A processor controls a display to show an adjustable region of interest (ROI) smaller than a field of view of the image sensor. Responsive to designated conditions, a frame is selected for diagnostic image processing to determine whether a feature of interest (FOI) is present in the frame. Responsive to an FOI being present in the frame but outside the ROI, a notification is created on the electronic display.

First claim

The invention claimed is:

1. An imaging scope system, comprising:
a shaft including a distal tip with a light emitter providing illumination light and an optical assembly including a wide-angle lens element;
an image sensor assembly including an image sensor configured to receive at least a portion of light focused through the optical assembly and produce output signals;
image forming circuitry adapted to receive the output signals and produce an image signal communicating a series of image frames;
a processor coupled to the image forming circuitry and configured to:
condition the image signal for presentation on an electronic display, the presentation including an adjustable region of interest (ROI) smaller than a field of view of the image sensor;
responsive to designated conditions, capture a frame from a series of image frames, by the image sensor, for diagnostic image processing, and evaluate results of the diagnostic image processing to determine whether a feature of interest (FOI) is present in the frame, wherein, during a time in which the frame is captured, causing the light emitter on the imaging scope to alter illumination light from a first state to a second state, and return to the first state following the capture of the frame; and

responsive to an FOI being present in the frame but outside the ROI, create a notification on the electronic display.

2. The system according to claim 1 , wherein spectral content of the second state is different than the spectral content of first state.

3. The system according to claim 2 , wherein the second state comprises illumination light appropriate to stimulate fluorescence in prepared tissue.

4. The system according to claim 3 , wherein the second state consists substantially of light from a spectral band spanning 350-450 nm.

5. The system according to claim 1 , wherein second state is of a higher intensity than the first state.

6. The system according to claim 1 , wherein the second state has a wider field of illumination than that of the first state.

7. The system according to claim 1 , wherein the processor is further configured to:
cause the selected frame not to be displayed in the presentation, and instead replacing the selected frame with imagery based on at least one frame prior to the selected frame.

8. The system according to claim 1 , wherein the notification indicates a relative direction from the ROI to the FOI.

9. The system according to claim 1 , wherein the diagnostic image processing comprises an artificial intelligence (AI) algorithm configured to detect cancerous or precancerous tumors or lesions.

10. The system according to claim 9 , wherein the AI algorithm comprises a trained convolutional neural network (CNN).

11. The system according to claim 1 , wherein the designated conditions include waiting at least 500 ms for a next image acquisition.

12. The system according to claim 1 , wherein the designated conditions include receiving a signal from a movement sensor on the imaging scope indicating that the imaging scope has moved more than a designated threshold.

13. The system according to claim 1 , wherein the designated conditions include a change in content of the series of image frames in the ROI above a predetermined threshold.

14. A method comprising:
receiving, at a processor, an image signal based on sensor data received from an image sensor of an imaging scope;
conditioning the image signal for presentation on an electronic display, the presentation including an adjustable region of interest (ROI) smaller than a field of view of the image sensor;
responsive to designated conditions, capturing a frame from a series of image frames, by the image sensor, for diagnostic image processing, wherein, during a time in which the frame is captured, causing a light emitter on the imaging scope to alter illumination light from a first state to a second state, and return to the first state following the capture of the frame;
performing the diagnostic image processing and evaluating results of the diagnostic image processing to determine whether a feature of interest (FOI) is present in the selected frame; and
responsive to the FOI being present in the selected frame but outside of the ROI, causing a notification to be displayed on the electronic display.

15. The method according to claim 14 , wherein spectral content of the second state is different than the spectral content of first state.

16. The method according to claim 15 , wherein the second state comprises illumination light appropriate to stimulate fluorescence in prepared tissue.

17. The method according to 16 , wherein the second state consists substantially of light from a spectral band spanning 350-450 nm.

18. The method according to claim 14 , wherein the second state is of a higher intensity than the first state.

19. The method according to claim 14 , wherein the second state has a wider field of illumination than that of the first state.

20. The method according to claim 14 , further comprising causing the selected frame not to be displayed in the presentation, and instead replacing the selected frame with imagery based on at least one frame prior to the selected frame.

21. The method according to claim 14 , wherein the notification indicates a relative direction from the ROI to the FOI.

22. The method according to claim 14 , wherein the diagnostic image processing comprises an artificial intelligence (AI) algorithm configured to detect cancerous or precancerous tumors or lesions.

23. The method according to claim 22 , wherein the AI algorithm comprises a trained convolutional neural network (CNN).

24. The method according to claim 14 , wherein the designated conditions include waiting at least 500 ms for a next image acquisition.

25. The method according to claim 14 , wherein the designated conditions include receiving a signal from a movement sensor on the imaging scope indicating that the imaging scope has moved more than a designated threshold.

26. The method according to claim 14 , wherein the designated conditions include a change in content of the series of image frames in the ROI above a predetermined threshold.

Assignees

• Storz Karl Imaging Inc

Inventors

• Duckett Iii George E

Classifications

• A61B1/00043

  provided with output arrangements · CPC title

• A61B1/000096

  using artificial intelligence · CPC title

• H04N5/2628

  Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title