Optimizing speech to text conversion and text summarization using a medical provider workflow model

Patent metadata

Field	Value
Publication number	US-11094322-B2
Application number	US-201916269795-A
Country	US
Kind code	B2
Filing date	Feb 7, 2019
Priority date	Feb 7, 2019
Publication date	Aug 17, 2021
Grant date	Aug 17, 2021

Abstract

A method, a system, and a computer program product are provided. Speech signals from a medical conversation between a medical provider and a patient are converted to text based on a first domain model associated with a medical scenario. The first domain model is selected from multiple domain models associated with a workflow of the medical provider. One or more triggers are detected, each of which indicates a respective change in the medical scenario. A corresponding second domain model is applied to the medical conversation to more accurately convert the speech signals to text in response to each of the detected one or more triggers. The corresponding second domain model is associated with a respective change in the medical scenario of the workflow of the medical provider. A clinical note is provided based on the text produced by converting the speech signals.

First claim

The invention claimed is:

1. A method of processing a medical conversation comprising:
converting to text, via a processor, speech signals from the medical conversation between a medical provider and a patient for a workflow of the medical provider based on a first domain model of a plurality of domain models, wherein the workflow includes a plurality of medical scenarios and the plurality of domain models are each trained for a corresponding medical scenario of the workflow, and wherein the first domain model is associated with a current medical scenario of the medical conversation;
detecting, via the processor during the converting to text, one or more triggers occurring during the medical conversation, each of the one or more triggers indicating a change to a different medical scenario of the workflow within the medical conversation;
in response to each of the detected one or more triggers, applying, via the processor during the converting to text, a corresponding second domain model of the plurality of domain models to the speech signals of the medical conversation pertaining to the different medical scenario indicated by the detected trigger to convert the speech signals pertaining to the different medical scenario indicated by the detected trigger from the medical conversation to text, wherein the corresponding second domain model is trained for the different medical scenario indicated by the detected trigger; and
providing, via the processor, a clinical note based on the text produced from the speech signals of the medical conversation.

2. The method of claim 1 , further comprising storing the speech signals from the medical conversation, wherein the applying the corresponding second domain model further comprises:
applying a plurality of second domain models to the stored speech signals based on the detected one or more triggers to convert the stored speech signals to text.

3. The method of claim 2 , wherein
the detecting of the one or more triggers is based on recognizing at least one of certain words, certain phrases and certain combinations of words in the medical conversation.

4. The method of claim 1 , further comprising:
learning, via the processor, one or more domain models based on previous interactions between a medical provider and a patient.

5. The method of claim 1 , wherein the plurality of domain models includes domain models at different levels of granularity for corresponding medical scenarios.

6. The method of claim 1 , wherein the plurality of domain models includes a domain model for use with a plurality of different medical scenarios.

7. The method of claim 1 , further comprising:
automatically performing, via the processor, an action included in the clinical note.

8. A system for processing a medical conversation, the system comprising:
at least one processor; and
at least one memory connected to the at least one processor, the at least one processor being configured to:
convert to text speech signals from the medical conversation between a medical provider and a patient for a workflow of the medical provider based on a first domain model of a plurality of domain models, wherein the workflow includes a plurality of medical scenarios and the plurality of domain models are each trained for a corresponding medical scenario of the workflow, and wherein the first domain model is associated with a current medical scenario of the medical conversation;
detect, during the converting to text, one or more triggers occurring during the medical conversation, each of the one or more triggers indicating a change to a different medical scenario of the workflow within the medical conversation;
in response to each of the detected one or more triggers, apply during the converting to text a corresponding second domain model of the plurality of domain models to the speech signals of the medical conversation pertaining to the different medical scenario indicated by the detected trigger to convert the speech signals pertaining to the different medical scenario indicated by the detected trigger from the medical conversation to text, wherein the corresponding second domain model is trained for the different in the medical scenario indicated by the detected trigger; and
provide a clinical note based on the text produced from the speech signals of the medical conversation.

9. The system of claim 8 , wherein the at least one processer is further configured to:
store the speech signals from the medical conversation, wherein the at least one processor being configured to apply the corresponding second domain model further comprises the at least one processor being configured to:
apply a plurality of second domain models to the stored speech signals based on the detected one or more triggers to convert the stored speech signals to text.

10. The system of claim 8 , wherein
the detecting of the one or more triggers is based on at least one of recognizing one or more certain words in the medical conversation and receiving signals from at least one sensor associated with a medical device.

11. The system of claim 8 , wherein the at least one processor is further configured to:
learn one or more domain models based on previous interactions between a medical provider and a patient.

12. The system of claim 8 , wherein the plurality of domain models includes domain models at different levels of granularity for corresponding medical scenarios.

13. The system of claim 8 , wherein the plurality of domain models includes a domain model for use with a plurality of different medical scenarios.

14. The system of claim 8 , wherein the clinical note is arranged according to a subjective, objective, assessment and plan format.

15. A computer program product for processing a medical conversation, the computer program product comprising at least one computer readable storage medium having computer readable program code embodied therewith for execution on at least one processor of a computing device, the computer readable program code being configured to:
convert to text speech signals from the medical conversation between a medical provider and a patient for a workflow of the medical provider based on a first domain model of a plurality of domain models, wherein the workflow includes a plurality of medical scenarios and the plurality of domain models are each trained for a corresponding medical scenario of the workflow, and wherein the first domain model is associated with a current medical scenario of the medical conversation;
detect, during the converting to text, one or more triggers occurring during the medical conversation, each of the one or more triggers indicating a change to a different medical scenario of the workflow within the medical conversation;
in response to each of the detected one or more triggers, apply during the converting to text a corresponding second domain model of the plurality of domain models to the speech signals of the medical conversation pertaining to the different medical scenario indicated by the detected trigger to convert the speech signals pertaining to the different medical scenario indicated by the detected trigger from the medical conversation to text, wherein the corresponding second domain model is trained for the different medical scenario indicated by the detected trigger; and
provide a clinical note based on the text produced from the speech signals of the medical conversation.

16. The computer program product of claim 15 , wherein the computer readable program code is further configured to:
store the speech signals from the medical conversation, wh

Assignees

• IBM

Inventors

• Lavery Andrew J
• Ng Kenney
• Picheny Michael
• Tang Paul C

Classifications

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G16H20/10

  relating to drugs or medications, e.g. for ensuring correct administration to patients · CPC title

• G10L15/183 Primary

  using context dependencies, e.g. language models · CPC title

• G16H10/40

  for data related to laboratory analysis, e.g. patient specimen analysis · CPC title

• G06F40/30

  Semantic analysis · CPC title