Active learning on biological sounds for determing presence of medical condition

Patent metadata

Field	Value
Publication number	US-2025099067-A1
Application number	US-202318372934-A
Country	US
Kind code	A1
Filing date	Sep 26, 2023
Priority date	Sep 26, 2023
Publication date	Mar 27, 2025
Grant date	—

Abstract

Methods and systems for training an audio-based machine learning model to predict a health condition based on biological sounds emitted by a person. Audio data corresponding to biological sounds produced by the person is generated from a microphone. The audio data is segmented into a plurality of segments, each segment associated with a respective sound event. An audio-based machine learning model is executed on the plurality of segments. The audio-based machine learning model is configured to output, for each segment, a label of a medical condition and an associated a confidence score. The model is trained via active learning, in which a subset of the plurality of segments are selected based on their confidence score being below a threshold, and provided to a human for annotation.

First claim

What is claimed is:

1 . A method for training an audio-based machine learning model with active learning, the method comprising:
receiving audio data corresponding to biological sounds produced by a body of a patient; segmenting the audio data into a plurality of segments; executing an audio-based machine learning model on the plurality of segments, wherein the audio-based machine learning model is configured to output, for each segment, a label of a medical condition and an associated a confidence score; storing the labels and associated confidence scores in storage; and training the audio-based machine learning model via active learning, wherein the training includes:
retrieving a subset of the plurality of segments from the storage;
receiving annotations from a human annotator, wherein the annotations are associated with medical conditions; and
training the audio-based machine learning model based on the annotations until convergence to yield a trained sound-based machine learning model configured to output a predicted medical condition associated with input biological sounds.

2 . The method of claim 1 , wherein the segmenting is performed via a pre-trained audio neural network (PANN), and each of the plurality of segments is associated with a respective audio event corresponding to an audio-based biomarker.

3 . The method of claim 1 , wherein the plurality of segments are each associated with breathing, the predicted medical condition includes asthma.

4 . The method of claim 1 , wherein the plurality of segments are each associated with heartbeats, the predicted medical condition includes a heart murmur.

5 . The method of claim 1 , wherein the subset of the plurality of segments are selected for retrieval based upon confidence scores associated with the plurality of segments being under a threshold.

6 . The method of claim 1 , wherein the microphone is attached to a stethoscope.

7 . The method of claim 1 , wherein the audio data is a continuous stream of audio data.

8 . A system comprising:
a processors; and a non-transitory memory coupled to the processor comprising instructions executable by the processor, the processor operable when executing the instructions to:
receive audio data generated from a microphone and corresponding to biological sounds produced by a body of a patient;
segment the audio data into a plurality of segments;
execute an audio-based machine learning model on the plurality of segments, wherein the audio-based machine learning model is configured to output, for each segment, a label of a medical condition and an associated a confidence score;
store the labels and associated confidence scores; and
train the audio-based machine learning model via active learning, wherein the training includes:
retrieving a subset of the plurality of segments;
receiving annotations from a human annotator, wherein the annotations are associated with medical conditions; and
training the audio-based machine learning model based on the annotations until convergence to yield a trained sound-based machine learning model configured to output a predicted medical condition.

9 . The system of claim 8 , wherein the audio data is a continuous stream of audio data.

10 . The system of claim 9 , wherein the segmenting of the audio data is performed via a pre-trained audio neural network (PANN), and each of the plurality of segments is associated with a respective audio event corresponding to an audio-based biomarker.

11 . The system of claim 8 , wherein the plurality of segments are each associated with breathing, the predicted medical condition includes asthma.

12 . The system of claim 8 , wherein the plurality of segments are each associated with heartbeats, the predicted medical condition includes a heart murmur.

13 . The system of claim 8 , wherein the subset of the plurality of segments are selected for retrieval based upon confidence scores associated with the plurality of segments being under a threshold.

14 . The system of claim 8 , wherein the microphone is attached to a stethoscope.

15 . A computer-readable non-transitory storage medium embodying software that is operable, when executed, to:
receive audio data generated from a microphone and corresponding to biological sounds produced by a body of a patient; segment the audio data into a plurality of segments, execute an audio-based machine learning model on the plurality of segments, wherein the audio-based machine learning model is configured to output, for each segment, a label of a medical condition and an associated a confidence score; store the labels and associated confidence scores; and train the audio-based machine learning model via active learning, wherein the training includes:
retrieving a subset of the plurality of segments;
receiving annotations from a human annotator, wherein the annotations are associated with medical conditions; and
training the audio-based machine learning model based on the annotations until convergence to yield a trained sound-based machine learning model configured to output a predicted medical condition.

16 . The computer-readable non-transitory storage medium of claim 15 ,
wherein the audio data is a continuous stream of audio data.

17 . The computer-readable non-transitory storage medium of claim 16 , wherein the segmenting of the audio data is performed via a pre-trained audio neural network (PANN), and each of the plurality of segments is associated with a respective audio event corresponding to an audio-based biomarker.

18 . The computer-readable non-transitory storage medium of claim 15 , wherein the plurality of segments are each associated with breathing, the predicted medical condition includes asthma.

19 . The computer-readable non-transitory storage medium of claim 15 , wherein the plurality of segments are each associated with heartbeats, the predicted medical condition includes a heart murmur.

20 . The computer-readable non-transitory storage medium of claim 15 , wherein the subset of the plurality of segments are selected for retrieval based upon confidence scores associated with the plurality of segments being under a threshold.

Assignees

• Bosch Gmbh Robert
• Allegheny Singer Res Institute
• Highmark Health

Inventors

• Ghaffarzadegan Shabnam
• Das Samarjit
• Bondi Luca
• Wu Ho-Hsiang
• Aracri Joseph
• Shields Kelly J
• Munir Sirajum

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title

• A61B7/04

  Electric stethoscopes · CPC title

• A61B5/082

  Evaluation by breath analysis, e.g. determination of the chemical composition of exhaled breath (A61B5/083, A61B5/091 take precedence) · CPC title

• G06N3/091

  Active learning · CPC title