System for transcribing and performing analysis on patient data

Patent metadata

Field	Value
Publication number	US-12295675-B2
Application number	US-202418439056-A
Country	US
Kind code	B2
Filing date	Feb 12, 2024
Priority date	Jan 4, 2022
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

Methods, apparatuses, and systems for transcribing and performing analysis on patient data are disclosed. Data is collected from one or more medical professionals as well as sensors and imaging devices positioned on or oriented towards a patient. An analysis is performed on the patient data and the data is presented to a medical professional via a verbal interface in a conversational manner, allowing the medical professional to provide additional data such as observations or instructions which may be used for further analysis or to perform actions related to the patient's care.

First claim

What is claimed is:

1. A surgical robotic system comprising:
a surgical robot configured to perform one or more surgical actions for a surgical procedure, wherein the surgical robot includes
a robotic arm, and
an end effector;

a voice-interactive transcription system configured to transcribe physician input associated with a physician, wherein the voice-interactive transcription system includes a plurality of transcription modules that are selectable for the surgical procedure;
a computing device communicatively coupled to the surgical robot and programed to intra-operatively perform a process comprising:
analyzing a body of a patient based on sensor data generated by a sensor, and
controlling the surgical robot to perform an additional surgical action based on the analysis of the body of the patient and a transcribed physician input from one of the plurality of transcription modules of the voice-interactive transcription system selected for the surgical procedure; and

outputting audible information to the physician, wherein the voice-interactive transcription system generates the audible information using computer-generated natural language.

2. The surgical robotic system of claim 1 , further comprising one or more microphones configured to receive speech input from the physician, wherein the computing device is programmed to control the surgical robot based on the speech input and a surgical plan for the surgical procedure.

3. The surgical robotic system of claim 1 , wherein the computing device includes one or more speakers for outputting the audible information.

4. The surgical robotic system of claim 1 , further comprising a console configured to control, via a direct connection or local network, operation of the surgical robot, wherein the console includes a display screen, one or more user-input devices, and the computing device.

5. The surgical robotic system of claim 1 , wherein the process further comprises:
monitoring live surgical procedures; and
training a machine learning model based on the live surgical procedures while the live surgical procedures are being performed.

6. The surgical robotic system of claim 1 , wherein the transcribed physician input comprises medical notes, wherein the process further comprises:
determining a diagnosis based on the medical notes.

7. The surgical robotic system of claim 1 , wherein the sensor data comprises medical images of the body of the patient received from an imaging device, wherein the process further comprises:
extracting an anomalous feature from the medical images; and
determining a diagnosis using machine learning performed on the anomalous feature.

8. The surgical robotic system of claim 1 , wherein the process further comprises:
for each of one or more medical conditions:
assigning a statistical probability that the patient has each medical condition using a machine learning model; and
selecting a medical condition having a greatest statistical probability.

9. The surgical robotic system of claim 8 , wherein the process further comprises:
selecting each of the one or more medical conditions having a statistical probability greater than a threshold statistical probability.

10. The surgical robotic system of claim 1 , wherein the process further comprises:
generating a predicted diagnosis and a predicted treatment for a previous patient based on historical patient data;
performing a comparison of the predicted diagnosis and the predicted treatment to an actual diagnosis and an actual treatment provided by a previous surgeon in the historical patient data; and
training a regression model based on the comparison.

11. A method comprising:
performing, using a surgical robot of a surgical system, one or more surgical actions for a surgical procedure, wherein the surgical robot includes
a robotic arm, and
an end effector;

transcribing, using a voice-interactive transcription system, one or more physician inputs from a physician, wherein the voice-interactive transcription system includes a plurality of transcription modules that are selectable for the surgical procedure;
intra-operatively performing, using a computing device communicatively coupled to the surgical robot, a process comprising:
analyzing a body of a patient based on sensor data generated by a sensor, and
controlling the surgical robot to perform an additional surgical action based on the analysis of the body of the patient and a transcribed physician input from one of the plurality of transcription modules of the voice-interactive transcription system selected for the surgical procedure; and

outputting audible information to the physician, wherein the voice-interactive transcription system generates the audible information using computer-generated natural language.

12. The method of claim 11 , further comprising:
receiving speech input from the physician using one or more microphones,
controlling, using the computing device, the surgical robot based on the speech input and a surgical plan for the surgical procedure.

13. The method of claim 11 , wherein the computing device includes one or more speakers for outputting the audible information.

14. The method of claim 11 , further comprising:
controlling, using a console, via a direct connection or local network, operation of the surgical robot, wherein the console includes a display screen, one or more user-input devices, and the computing device.

15. The method of claim 11 , further comprising:
monitoring live surgical procedures; and
training a machine learning model based on the live surgical procedures while the live surgical procedures are being performed.

16. The method of claim 11 , wherein the transcribed physician input comprises medical notes, the method further comprising:
determining a diagnosis based on the medical notes.

17. The method of claim 11 , wherein the sensor data comprises medical images of the body of the patient received from an imaging device, the method further comprising:
extracting an anomalous feature from the medical images; and
determining a diagnosis using machine learning performed on the anomalous feature.

18. The method of claim 11 , further comprising:
assigning a statistical probability that the patient has each medical condition using a machine learning model; and
selecting a medical condition having a greatest statistical probability.

19. The method of claim 11 , further comprising:
generating a predicted diagnosis and a predicted treatment for a previous patient based on historical patient data;
performing a comparison of the predicted diagnosis and the predicted treatment to an actual diagnosis and an actual treatment provided by a previous surgeon in the historical patient data; and
training a regression model based on the comparison.

20. A non-transitory computer-readable medium storing instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
performing, using a surgical robot of a surgical system, one or more surgical actions for a surgical procedure, wherein the surgical robot includes
a robotic arm, and
an end effector;

transcribing, using a voice-interactive transcription system, one or more physician inputs from a physician, wherein the voice-interactive transcription system includes a plurality of transcription modules that are selectable for the surgical procedure;
intra-operatively performing, using a computing device communicatively coupled to the surgical

Assignees

• Ix Innovation Llc

Inventors

• Roh Jeffrey
• Esterberg Justin
• Cronin John
• Cronin Seth
• Baker Michael John

Classifications

• G16H10/60

  for patient-specific data, e.g. for electronic patient records · CPC title

• G06T2207/30004

  Biomedical image processing · CPC title

• G06V2201/03

  Recognition of patterns in medical or anatomical images · CPC title

• G16B40/20

  Supervised data analysis · CPC title

• G16B20/20

  Allele or variant detection, e.g. single nucleotide polymorphism [SNP] detection · CPC title