Intelligent classification of regions of interest of an organism from multispectral video streams using perfusion models

Patent metadata

Field	Value
Publication number	US-11145052-B2
Application number	US-201916394596-A
Country	US
Kind code	B2
Filing date	Apr 25, 2019
Priority date	Apr 25, 2019
Publication date	Oct 12, 2021
Grant date	Oct 12, 2021

Abstract

Embodiments for implementing intelligent classification of region of interest in an organism in a computing environment by a processor. Time series data of a contrast agent in one or more regions of interest captured from multispectral image streams may be collected. The one or more regions of interest may be classified into one of a plurality of classes by applying one or more perfusion models, representing spatio-temporal behavior of the contrast agent reflected by the time series data, and by using a machine learning operation.

First claim

The invention claimed is:

1. A method for implementing intelligent classification of regions of interest of an organism by a processor, comprising:
collecting time series data of a contrast agent, injected into a patient and perfusing through one or more regions of interest, from multispectral image streams;
classifying the one or more regions of interest into one of a plurality of classes by using a machine learning operation to apply one or more perfusion models representing spatio-temporal behavior of the contrast agent reflected by the time series data, wherein the classification is performed by the machine learning operation in real-time and eliminates a waiting period, from injection to a stationary phase in which the contrast agent is persisted or flushed from the one or more regions of interest, otherwise required to perform a medical diagnosis.

2. The method of claim 1 , further including estimating one or more parameters of the one or more perfusion models for classifying the one or more regions of interest.

3. The method of claim 1 , further including:
receiving, in real-time, the multispectral image streams of the one or more regions of interest for a selected period of time from an image capturing device;
collecting user profile data; or
collecting a corpus of labels for labeling the multispectral image streams.

4. The method of claim 1 , further including labeling the one of the plurality of classes of the one or more regions of interest in the multispectral image streams, wherein the one of the plurality of classes represents at least a predicted medical diagnosis of the one or more regions of interest.

5. The method of claim 1 , further including assigning a confidence score to the one of the plurality of classes of the one or more regions of interest.

6. The method of claim 1 , further including identifying a fluorescence intensity level of the contrast agent captured from the multispectral image streams, wherein the spatio-temporal behavior includes the fluorescence intensity level and the fluorescence intensity level represents a concentration level of the contrast agent in the one or more regions of interest.

7. The method of claim 1 , further including initiating the machine learning operation to train or retrain the one or more perfusion models according to a repository of plurality of multispectral image streams, a corpus of classes or labels of each of the plurality of multispectral image streams, a plurality of time series data, labeled regions of interest, patient profile data, or a combination thereof.

8. A system for implementing intelligent classification of region of interest of an organism, comprising:
one or more computers with executable instructions that when executed cause the system to:
collect time series data of a contrast agent, injected into a patient and perfusing through one or more regions of interest, from multispectral image streams;
classify the one or more regions of interest into one of a plurality of classes by using a machine learning operation to apply one or more perfusion models representing spatio-temporal behavior of the contrast agent reflected by the time series data wherein the classification is performed by the machine learning operation in real-time and eliminates a waiting period, from injection to a stationary phase in which the contrast agent is persisted or flushed from the one or more regions of interest, otherwise required to perform a medical diagnosis.

9. The system of claim 8 , wherein the executable instructions further estimate one or more parameters of the one or more perfusion models for classifying the one or more regions of interest.

10. The system of claim 8 , wherein the executable instructions further:
receive, in real-time, the multispectral image streams of the one or more regions of interest for a selected period of time from an image capturing device;
collect user profile data; or
collect a corpus of labels for labeling the multispectral image streams.

11. The system of claim 8 , wherein the executable instructions further label the one of the plurality of classes of the one or more regions of interest in the multispectral image streams, wherein the one of the plurality of classes represents at least a predicted medical diagnosis of the one or more regions of interest.

12. The system of claim 8 , wherein the executable instructions further assign a confidence score to the one of the plurality of classes of the one or more regions of interest.

13. The system of claim 8 , wherein the executable instructions further identify a fluorescence intensity level of the contrast agent captured from the multispectral image streams, wherein the spatio-temporal behavior includes the fluorescence intensity level and the fluorescence intensity level represents a concentration level of the contrast agent in the one or more one or more regions of interest.

14. The system of claim 8 , wherein the executable instructions further initiate the machine learning operation to train or retrain the one or more perfusion models according to a repository of a plurality of multispectral image streams, a corpus of labels of each of the plurality of multispectral image streams, a plurality of time series data, labeled regions of interest, patient profile data, or a combination thereof.

15. A computer program product for implementing intelligent classification of region of interest of an organism by a processor, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:
an executable portion that collects time series data of a contrast agent, injected into a patient and perfusing through one or more regions of interest, from multispectral image streams;
an executable portion that classifies the one or more regions of interest into one of a plurality of classes by using a machine learning operation to apply one or more perfusion models representing spatio-temporal behavior of the contrast agent reflected by the time series data, wherein the classification is performed by the machine learning operation in real-time and eliminates a waiting period, from injection to a stationary phase in which the contrast agent is persisted or flushed from the one or more regions of interest, otherwise required to perform a medical diagnosis.

16. The computer program product of claim 15 , further including an executable portion that estimates one or more parameters of the one or more perfusion models for classifying the one or more regions of interest.

17. The computer program product of claim 15 , further including an executable portion that:
receives, in real-time, the multispectral image streams of the one or more regions of interest for a selected period of time from an image capturing device;
collects user profile data; or
collects a corpus of labels for labeling the multispectral image streams.

18. The computer program product of claim 15 , further including an executable portion that:
labels the one of the plurality of classes of the one or more regions of interest in the multispectral image streams, wherein the one of the plurality of classes represents at least a predicted medical diagnosis of the one or more regions of interest; and
assigns a confidence score to the one of the plurality of classes of the one or more regions of interest.

19. The computer program product of claim 15 , further including an executable portion that a fluores

Assignees

• IBM
• Univ College Dublin School Of Medicine & Medical Science

Inventors

• Epperlein Jonathan
• Zhuk Sergiy
• Nair Rahul
• Macaonghusa Pol

Classifications

• G06T7/0016 Primary

  involving temporal comparison · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06F18/24

  Classification techniques · CPC title