Systems and methods for processing electronic medical images to determine object size

Patent metadata

Field	Value
Publication number	US-12175673-B2
Application number	US-202318455060-A
Country	US
Kind code	B2
Filing date	Aug 24, 2023
Priority date	Nov 18, 2019
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

Systems and methods for processing electronic images from a medical device comprise receiving an image frame from the medical device, and determining a first color channel and a second color channel in the image frame. A location of an electromagnetic beam halo may be identified by comparing the first color channel and second color channel. Edges of an electromagnetic beam may be determined based on the electromagnetic beam halo, and size metrics of the electromagnetic beam may be determined based on the edges of the electromagnetic beam. A visual indicator on the image frame may be displayed based on the size metrics of the electromagnetic beam.

First claim

We claim:

1. A method for processing electronic medical images, comprising:
receiving an image frame of an object from a medical imaging device;
identifying, within the image frame, a halo associated with an electromagnetic beam projected onto the object while the image frame was captured by the medical imaging device;
determining size metrics of the electromagnetic beam based on the halo; and
determining size metrics of the object based on the size metrics of the electromagnetic beam.

2. The method of claim 1 , further comprising determining size metrics of an exit channel through which the object is to be passed based on the size metrics of the electromagnetic beam.

3. The method of claim 2 , further comprising generating a visual indicator for display on the image frame, wherein the visual indicator represents the size metrics of the exit channel to visually indicate whether the object is able to be passed through the exit channel.

4. The method of claim 2 , further comprising providing one or more rulers for display on the image frame based on the size metrics determined for one or more of the electromagnetic beam, the object, or the exit channel.

5. The method of claim 1 , wherein determining the size metrics of the electromagnetic beam based on the halo comprises:
determining edges of the electromagnetic beam based on the halo; and
determining the size metrics of the electromagnetic beam based on the edges.

6. The method of claim 5 , further comprising:
determining an approximated electromagnetic beam based on the halo;
distinguishing the approximated electromagnetic beam from image artifacts based on a size of the approximated electromagnetic beam relative to the image artifacts;
fitting a shape to the approximated electromagnetic beam; and
determining approximated size metrics for the shape fitted to the approximated electromagnetic beam.

7. The method of claim 6 , further comprising:
extracting features from the image frame within a predetermined distance of one or more of the electromagnetic beam or the halo;
providing, as input to a trained machine learning system, the extracted features and the approximated size metrics for the shape fitted to the approximated electromagnetic beam; and
receiving, as output of the trained machine learning system, the size metrics of the electromagnetic beam, wherein the size metrics of the electromagnetic beam are altered from the approximated size metrics based on weights applied by the trained machine learning system.

8. The method of claim 1 , wherein identifying the halo comprises:
determining a first color channel and a second color channel in the image frame; and
comparing the first color channel to the second color channel to identify the halo.

9. The method of claim 8 , wherein comparing the first color channel to the second color channel comprises:
determining that the first color channel depicts the electromagnetic beam and the halo;
determining that the second color channel depicts the electromagnetic beam; and
subtracting the second color channel from the first color channel, or adding an inverse of the first color channel to the second color channel to identify the halo.

10. The method of claim 9 , wherein the first color channel is determined so as to match a predetermined color of the electromagnetic beam, and wherein the second color channel is determined so as to not match the predetermined color of the electromagnetic beam.

11. A method for processing electronic medical images, comprising:
receiving an image frame from a medical imaging device, the image frame including an object onto which an electromagnetic beam is projected by the medical imaging device, the projection causing a halo to be formed in association with the electromagnetic beam;
distinguishing the electromagnetic beam from the halo within the image frame;
determining size metrics of the electromagnetic beam; and
determining relative size metrics of the object based on the size metrics of the electromagnetic beam.

12. The method of claim 11 , further comprising determining relative size metrics of an exit channel associated with the medical imaging device through which the object is to be passed based on the size metrics of the electromagnetic beam.

13. The method of claim 12 , further comprising generating a visual indicator for display on the image frame, wherein the visual indicator represents the relative size metrics of the exit channel to visually indicate whether the object is able to be passed through the exit channel.

14. The method of claim 11 , wherein distinguishing the electromagnetic beam from the halo within the image frame comprises:
identifying a location of the halo within the image frame; and
determining edges of the electromagnetic beam based on the halo.

15. The method of claim 14 , wherein determining the size metrics of the electromagnetic beam comprises:
fitting a shape to an approximated electromagnetic beam based on the edges; and
determining approximated size metrics for the shape fitted to the approximated electromagnetic beam.

16. The method of claim 15 , further comprising:
extracting features from the image frame within a predetermined distance of one or more of the electromagnetic beam or the halo;
providing, as input to a trained machine learning system, the extracted features and the approximated size metrics for the shape fitted to the approximated electromagnetic beam; and
receiving, as output of the trained machine learning system, the size metrics of the electromagnetic beam, wherein the size metrics of the electromagnetic beam are altered from the approximated size metrics based on weights applied by the trained machine learning system.

17. The method of claim 14 , wherein identifying the location of the halo within the image frame comprises:
determining a first color channel in the image frame that depicts the electromagnetic beam and the halo;
determining a second color channel in the image frame that depicts the electromagnetic beam; and
comparing the first color channel to the second color channel to identify the location of the halo.

18. A method for processing electronic medical images, comprising:
receiving, from a medical imaging device positioned in a body lumen, an image frame of an object to be removed from the body lumen, wherein the image frame is captured by the medical imaging device while an electromagnetic beam is being projected by the medical imaging device onto the object;
identifying a halo associated with the electromagnetic beam within the image frame;
determining size metrics of the electromagnetic beam using the halo; and
based on the size metrics of the electromagnetic beam, determining size metrics of the object relative to size metrics of an exit channel associated with the medical imaging device through which the object is to be removed from the body lumen.

19. The method of claim 18 , further comprising:
generating a visual indicator for display on the image frame, wherein the visual indicator represents the size metrics of the exit channel to visually indicate whether the object is able to be removed from the body lumen through the exit channel.

20. The method of claim 19 , wherein the medical imaging device is a ureteroscope, and the object to be removed is a kidney stone.

Assignees

• Boston Scient Scimed Inc

Inventors

• Chen Longquan
• Demisse Mussie Teshome
• Fusman Judith
• Silva Aaron
• Cahill Andrea
• Gyllenhaal Evan

Classifications

• G06T11/23

  using straight lines or curves · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06V2201/03

  Recognition of patterns in medical or anatomical images · CPC title

• G06T2207/30004

  Biomedical image processing · CPC title

• G06T2207/10068

  Endoscopic image · CPC title