Multi modality brain mapping system (MBMS) using artificial intelligence and pattern recognition

Patent metadata

Field	Value
Publication number	US-9754371-B2
Application number	US-201514815768-A
Country	US
Kind code	B2
Filing date	Jul 31, 2015
Priority date	Jul 31, 2014
Publication date	Sep 5, 2017
Grant date	Sep 5, 2017

Abstract

A Multimodality Brain Mapping System (MBMS), comprising one or more scopes (e.g., microscopes or endoscopes) coupled to one or more processors, wherein the one or more processors obtain training data from one or more first images and/or first data, wherein one or more abnormal regions and one or more normal regions are identified; receive a second image captured by one or more of the scopes at a later time than the one or more first images and/or first data and/or captured using a different imaging technique; and generate, using machine learning trained using the training data, one or more viewable indicators identifying one or abnormalities in the second image, wherein the one or more viewable indicators are generated in real time as the second image is formed. One or more of the scopes display the one or more viewable indicators on the second image.

First claim

What is claimed is:

1. A system, comprising one or more scopes coupled to one or more processors, wherein:
the one or more processors:
obtain training data from one or more first images and/or first data, wherein one or more abnormal regions and one or more normal regions are identified;
receive a second image captured by one or more of the scopes at a later time than the one or more first images and/or first data and/or captured using a different imaging technique; and
generate, using machine learning trained using the training data, one or more viewable indicators identifying one or more abnormalities in the second image, wherein the one or more viewable indicators are generated in real time as the second image is formed; and

one or more of the scopes display the one or more viewable indicators on the second image.

2. The system of claim 1 , wherein:
one or more of the processors comprise one or more multi-modality data processors; and
the multi-modality data processors register at least two of the first images and/or first data obtained from biopsy, Infrared Imaging, Ultraviolet Imaging, Diffusion Tensor Imaging (DTI), Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Brain Mapping Ultrasound, cellular data, molecular data, genomic data, optical imaging, and Functional MRI (FMRI), to form a registered image and/or patient data; and
one or more of the processors receive input that identifies or marks the one or more abnormal and one or more normal regions in the registered image and/or patient data.

3. The system of claim 1 , wherein the one or more first images comprise a pre-operative image, an intra-operative image, and/or a post operative image of one or more patients.

4. The system of claim 1 , wherein:
the one or more scopes comprise one or more first scopes and a second scope; and
the one or more first scopes capture the one or more first images of one or more patients and the second scope captures the second image of a different patient.

5. The system of claim 4 , further comprising:
a cloud and/or parallel computing system wherein the training data obtained from the one or more first images captured in the one or more first scopes is shared so that the machine learning learns from the training data to identify the one or more abnormalities in the second image of the different patient.

6. The system of claim 1 , wherein the processors that predict growth of the one or more abnormalities in the second image from predictive modeling of, and/or pattern recognition in, the training data.

7. The system of claim 1 , wherein one or more of the processors:
represent the one or more abnormal regions with first feature vectors defining first coordinates in a feature space;
represent the one or more normal regions with second feature vectors defining second coordinates in the feature space, wherein the feature space is selected such that at least some of the first coordinates and at least some of the second coordinates are on opposite sides of a hyper-plane in the feature space;
map an image region of the second image to one or more image coordinates in the feature space;
classify one or more of the image coordinates as one or more abnormal coordinates depending on one or more factors, including:
which side of the hyper-plane the one or more image coordinates lie; and/or
proximity of the one or more image coordinates to the first coordinates and/or the hyper-plane; and

indicate the image region as an abnormal image region if the image region is mapped to one or more of the abnormal coordinates according to the map.

8. The system of claim 1 , wherein the one or more processors implement a support vector machine.

9. The system of claim 1 , wherein:
one or more of the scopes comprise one or more microscopes and/or one or more endoscopes including an optical imaging system for capturing the second image; and
the second image comprises an optical image.

10. The system of claim 1 , wherein:
one or more of the one or more scopes comprise a surgical scope capturing the second image during a surgical procedure on a patient, and
the one or more viewable indicators enable the one or more abnormalities to be surgically removed from the patient, during the surgical procedure, with increased precision by reducing damage to normal tissue.

11. A method for identifying one or more abnormalities in an image, comprising:
obtaining training data from one or more first images and/or first data, wherein one or more abnormal regions and one or more normal regions in the one or more first images and/or first data are identified;
receiving a second image of the tissue captured at a later time than the one or more first images and/or first data and/or using a different imaging technique; and
generating, using machine learning trained using the training data, one or more viewable indicators identifying one or more abnormalities in the second image, wherein the one or more viewable indicators are generated in real time as the second image is formed.

12. The method of claim 11 , further comprising:
registering at least two of the first images and/or first data obtained from biopsy, Infrared Imaging, Ultraviolet Imaging Diffusion Tensor Imaging (DTI), Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Brain Mapping Ultrasound, cellular data, molecular data, and genomic data, optical imaging, and Functional MRI (FMRI), to form a registered image and/or patient data; and
receiving input that identifies or marks the one or more abnormal and one or more normal regions in the registered image and/or patient data.

13. The method of claim 12 , wherein the one or more first images comprise a pre-operative image and/or a post operative image of a patient.

14. The method claim 11 , further comprising:
obtaining the one or more first images of one or more patients captured with one or more first scopes; and
obtaining the second image of a different patient captured with a second scope.

15. The method of claim 14 , further comprising connecting the one or more first scope and the second scope using a cloud and/or parallel computing system wherein the training data obtained from the one or more first images captured in the one or more first scopes is shared so that the machine learning learns from the training data to identify the one or more abnormalities in the second image of the different patient.

16. The method of claim 11 , wherein the machine learning predicts growth of the one or more abnormal cells in the second image from predictive modeling of, and/or pattern recognition in, the training data.

17. The method of claim 11 , further comprising:
representing the one or more abnormal regions with first feature vectors defining first coordinates in a feature space;
representing the one or more normal regions with second feature vectors defining second coordinates in the feature space, wherein the feature space is selected such that at least some of the first coordinates and at least some of the second coordinates are on opposite sides of a hyper-plane in the feature space;
mapping an image region of the second image to one or more image coordinates in the feature space;
classifying one or more of the image coordinates as one or more abnormal coordinates depending on one or more factors, including:
which side of the hyper-plane the one or more image coordinates lie; and/or
proximity of the one or more image coordinates to the first coordinates and/or the hyper-plane; and

indicating the image region as an abnormal image region if the image regio

Assignees

• California Inst Of Techn
• Int Brain Mapping And Intra-Operative Surgical Planning Found

Inventors

• Kateb Babak
• Nikzad Shouleh

Classifications

• G06T2207/10132

  Ultrasound image · CPC title

• G06T2207/10092

  Diffusion tensor magnetic resonance imaging [DTI] · CPC title

• G06T2207/10088

  Magnetic resonance imaging [MRI] · CPC title

• G06T2207/10048

  Infrared image · CPC title

• G01R33/5608

  Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels (image data processing or generation, in general G06T) · CPC title