Method for classifying images

Patent metadata

Field	Value
Publication number	US-12524989-B2
Application number	US-202217670082-A
Country	US
Kind code	B2
Filing date	Feb 11, 2022
Priority date	Feb 12, 2021
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

A method includes classifying, via a computational model, images of a source image stream as valid images or invalid images based on whether the images include biological tissue or a surgical tool; and generating a condensed image stream that includes the valid images. Another method includes classifying input images as valid images or invalid images using: a clustering algorithm that classifies each of the input images into either a first group or a second group and using labels that indicate whether the input images include a surgical tool. The method also includes training a computational model to identify the valid images based on whether the valid images include biological tissue or a surgical tool, or whether the valid images have at least a threshold level of clarity.

First claim

What is claimed is:

1 . A method comprising:
classifying, via a computational model, images of a source image stream as valid images or invalid images based on whether the images include biological tissue or a surgical tool, wherein classifying the images comprises identifying the valid images based on whether the valid images have at least a threshold level of clarity; generating a condensed image stream that includes the valid images; generating feature vectors for the valid images, wherein the feature vectors each characterize one of the valid images based on multiple criteria; identifying a subset of the valid images that corresponds to a phase of a surgical procedure by comparing the feature vectors; and generating metadata that identifies the subset of the valid images within the condensed image stream.

2 . The method of claim 1 , wherein the computational model was trained using input images labeled, by a clustering algorithm, as belonging to either a first group or a second group based on a hue, a saturation, or a brightness of the input images.

3 . The method of claim 1 , wherein the computational model was trained using input images labeled as including a surgical tool.

4 . The method of claim 1 , wherein the computational model was trained using input images labeled by a feature detection algorithm.

5 . The method of claim 1 , wherein classifying the images comprises classifying the images based on whether the images include the biological tissue and whether the images include a surgical tool.

6 . The method of claim 1 , wherein classifying the images comprises identifying the valid images based on the valid images including the biological tissue.

7 . The method of claim 6 , wherein identifying the valid images comprises identifying the valid images based on a level of a hue within the valid images that is associated with the biological tissue.

8 . The method of claim 1 , wherein classifying the images comprises identifying the valid images based on the valid images including a surgical tool.

9 . The method of claim 1 , wherein identifying the valid images based on whether the valid images have at least the threshold level of clarity comprises determining that the valid images have at least a threshold amount of features.

10 . The method of claim 1 , wherein comparing the feature vectors comprises applying a low pass filter to a pair of the feature vectors representing a pair of the valid images that are consecutive in time within the condensed image stream.

11 . The method of claim 1 , wherein comparing the feature vectors comprises identifying a distance between a consecutive pair of the feature vectors that exceeds a threshold distance.

12 . The method of claim 11 , wherein the consecutive pair is a first consecutive pair, and wherein comparing the feature vectors further comprises:
identifying a second distance between a second consecutive pair of the feature vectors that exceeds a threshold distance, wherein the second consecutive pair chronologically follows the first consecutive pair; and confirming that a duration separating the first consecutive pair and the second consecutive pair exceeds a threshold duration.

13 . The method of claim 12 , wherein the threshold duration is a first threshold duration, wherein comparing the feature vectors further comprises:
identifying a third distance between a third consecutive pair of the feature vectors that exceeds a threshold distance, wherein the third consecutive pair chronologically follows the second consecutive pair; and confirming that a second duration separating the second consecutive pair and the third consecutive pair exceeds the first threshold duration.

14 . The method of claim 11 , wherein the threshold distance is a first threshold distance and the consecutive pair is a first consecutive pair, wherein comparing the feature vectors further comprises identifying a second distance between a second consecutive pair of the feature vectors that exceeds a second threshold distance that is greater than the first threshold distance, wherein the second consecutive pair chronologically follows the first consecutive pair.

15 . The method of claim 1 , further comprising:
receiving user input indicating a quantity of subsets, wherein identifying the subset of the valid images comprises identifying subsets of the valid images equal in number to the quantity of subsets.

16 . The method of claim 1 , further comprising:
evaluating, for each image of the subset, distances between the image and each other image of the subset; and identifying a sub-segment of the subset of the valid images having a predetermined number of images and that has a minimum sum, a minimum median, or a minimum average of the distances.

17 . The method of claim 1 , wherein the computational model was trained using input images labeled, by a clustering algorithm, as belonging to either a first group or a second group based on a hue of the input images.

18 . A computing device comprising:
one or more processors; and a computer readable medium storing instructions that, when executed by the one or more processors, cause the computing device to perform functions comprising: classifying, via a computational model, images of a source image stream as valid images or invalid images based on whether the images include biological tissue or a surgical tool, wherein classifying the images comprises identifying the valid images based on whether the valid images have at least a threshold level of clarity; generating a condensed image stream that includes the valid images; generating feature vectors for the valid images, wherein the feature vectors each characterize one of the valid images based on multiple criteria; identifying a subset of the valid images that corresponds to a phase of a surgical procedure by comparing the feature vectors; and generating metadata that identifies the subset of the valid images within the condensed image stream.

19 . The computing device of claim 18 , wherein the computational model was trained using input images labeled, by a clustering algorithm, as belonging to either a first group or a second group based on a hue of the input images.

20 . A non-transitory computer readable medium storing instructions that, when executed by a computing device, cause the computing device to perform functions comprising:
classifying, via a computational model, images of a source image stream as valid images or invalid images based on whether the images include biological tissue or a surgical tool, wherein classifying the images comprises identifying the valid images based on whether the valid images have at least a threshold level of clarity; generating a condensed image stream that includes the valid images; generating feature vectors for the valid images, wherein the feature vectors each characterize one of the valid images based on multiple criteria; identifying a subset of the valid images that corresponds to a phase of a surgical procedure by comparing the feature vectors; and generating metadata that identifies the subset of the valid images within the condensed image stream.

Assignees

• Univ Washington
• Seattle Childrens Hospital

Inventors

• Adidharma Lingga
• Bly Randall
• Young Christopher
• Hannaford Blake
• Humphreys Ian
• Abuzeid Al-Waleed M
• Ferreira Manuel
• Moe Kristen S
• Li Yangming
• King Daniel

Classifications

• G06V10/762

  using clustering, e.g. of similar faces in social networks · CPC title

• G06T2207/30056

  Liver; Hepatic · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title

• G06T2207/10068

  Endoscopic image · CPC title

• G06V2201/03

  Recognition of patterns in medical or anatomical images · CPC title