Generating high quality training data collections for training artificial intelligence models

Patent metadata

Field	Value
Publication number	US-2023342913-A1
Application number	US-202217660717-A
Country	US
Kind code	A1
Filing date	Apr 26, 2022
Priority date	Apr 26, 2022
Publication date	Oct 26, 2023
Grant date	—

Abstract

Techniques are described for generating high quality training data collections for training artificial intelligence (AI) models in the medical imaging domain. A method embodiment comprises receiving, by a system comprising processor, input indicating a clinical context associated with usage of a medical image dataset, and selecting, by the system, one or more data scrutiny metrics for filtering the medical image dataset based on the clinical context. The method further comprises applying, by the system, one or more image processing functions to the medical image dataset to generate metric values of the one or more data scrutiny metrics for respective medical images included in the medical image dataset, filtering, by the system, the medical image dataset into one or more subsets based on one or more acceptability criteria for the metric values.

First claim

What is claimed is:

1 . A system, comprising:
a memory that stores computer executable components; and a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
a clinical criteria selection component that receives first input indicating a clinical context associated with usage of a medical image dataset;
a scrutiny criteria selection component that selects one or more data scrutiny metrics for filtering the medical image dataset based on the clinical context;
an image processing component that applies one or more image processing functions to the medical image dataset to generate metric values of the one or more data scrutiny metrics for respective medical images included in the medical image dataset; and
a filtering component that filters the medical image dataset into one or more subsets based on one or more acceptability criteria for the metric values.

2 . The system of claim 1 , wherein the first input indicates one or more clinical inferencing tasks for training one or more machine learning models to perform on the one or more subsets, and wherein the computer executable component further comprise:
a training data curation component that stores the one or more subsets in corresponding training data collections for training the one or more machine learning models to perform the one or more clinical inferencing tasks.

3 . The system of claim 2 , wherein the computer executable components further comprise:
a training component that trains the one or more machine learning models using the one or more subsets.

4 . The system of claim 1 , wherein the first input indicates one or more clinical inferencing tasks for training one or more machine learning models to perform on the one or more subsets, wherein the clinical criteria selection component further receives second input identifying one or more anatomical regions of interest relevant to the one or more clinical inferencing tasks, and wherein the filtering component further filters the medical image dataset into the one or more subsets based on whether the respective medical images depict the one or more anatomical regions of interest.

5 . The system of claim 1 , wherein the computer executable components further comprise:
a visualization component that generates one or more graphical visualizations representative of the metric values for the respective medical images; and a rendering component that renders the one or more graphical visualizations via an interactive graphical user interface.

6 . The system of claim 5 , wherein the acceptability criterion comprises acceptable values for the one or more metric values and wherein the one or more graphical visualizations distinguish the one or more subsets associated with the acceptable values from outlier images of the medical image dataset associated with unacceptable values.

7 . The system of claim 5 , wherein the interactive graphical user interface provides for receiving the first input and receiving additional input manually defining the one or more data scrutiny metrics and the one or more acceptability criteria.

8 . The system of claim 7 , wherein the one or more data scrutiny metrics comprise two or more data scrutiny metrics and wherein the interactive graphical user interface further provides for defining the acceptability criteria based on individual data scrutiny metrics of the two or more data scrutiny metrics and combinations of the two or more data scrutiny metrics and generating the one or more subsets based on individual data scrutiny metrics of the two or more data scrutiny metrics and combinations of the two or more data scrutiny metrics.

9 . The system of claim 1 , wherein the one or more data scrutiny metrics comprise one or more medical image quality metrics.

10 . The system of claim 9 , wherein the one or more medical image quality metrics are selected from the group consisting of: signal to noise ratio, peak signal to noise ratio, mean square error, structural similarity index, feature similarity index, variance inflation factor and Laplacian loss.

11 . A method comprising:
receiving, by a system comprising a processor, first input indicating a clinical context associated with usage of a medical image dataset; selecting, by the system, one or more data scrutiny metrics for filtering the medical image dataset based on the clinical context; applying, by the system, one or more image processing functions to the medical image dataset to generate metric values of the one or more data scrutiny metrics for respective medical images included in the medical image dataset; and filtering, by the system, the medical image dataset into one or more subsets based on one or more acceptability criteria for the metric values.

12 . The method of claim 11 , wherein the first input indicates one or more clinical inferencing tasks for training one or more machine learning models to perform on the one or more subsets, and wherein the method further comprises:
storing, by the system, the one or more subsets in corresponding training data collections for training the one or more machine learning models to perform the one or more clinical inferencing tasks.

13 . The method of claim 12 , wherein the computer executable components further comprise:
training, by the system, the one or more machine learning models using the one or more subsets.

14 . The method of claim 11 , wherein the first input indicates one or more clinical inferencing tasks for training one or more machine learning models to perform on the one or more subsets, and wherein the method further comprises:
receiving, by the system, second input identifying one or more anatomical regions of interest relevant to the one or more clinical inferencing tasks, and wherein the filtering comprises filtering the medical image dataset into the one or more subsets based on whether the respective medical images depict the one or more anatomical regions of interest.

15 . The method of claim 11 , further comprising:
generating, by the system, one or more graphical visualizations representative of the metric values for the respective medical images; and rendering, by the system, the one or more graphical visualizations via an interactive graphical user interface.

16 . The method of claim 15 , wherein the acceptability criterion comprises acceptable values for the one or more metric values and wherein the one or more graphical visualizations distinguish the one or more subsets associated with the acceptable values from outlier images of the medical image dataset associated with unacceptable values.

17 . The method of claim 15 , wherein the interactive graphical user interface provides for receiving the first input and receiving additional input manually defining the one or more data scrutiny metrics and the one or more acceptability criteria.

18 . The system of claim 17 , wherein the one or more data scrutiny metrics comprise two or more data scrutiny metrics and wherein the interactive graphical user interface further provides for defining the acceptability criteria based on individual data scrutiny metrics of the two or more data scrutiny metrics and combinations of the two or more data scrutiny metrics and generating the one or more subsets based on individual data scrutiny metrics of the two or more data scrutiny metrics and combinations of the two or more data scrutiny metrics.

19 . A machine-readable storage medium, comprising executable instructions that

Assignees

• Ge Prec Healthcare Llc

Inventors

• Patil Mahendra Madhukar
• Mullick Rakesh
• Chatterjee Sudhanya
• Hashmi Syed Asad
• Shanbhag Dattesh Dayanand
• Anand Deepa
• Joel Suresh Emmanuel Devadoss

Classifications

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06N20/00

  Machine learning · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V2201/03

  Recognition of patterns in medical or anatomical images · CPC title

• G06N3/08 Primary

  Learning methods · CPC title