Methods for arthroscopic video analysis and devices therefor

Patent metadata

Field	Value
Publication number	US-11810356-B2
Application number	US-202217891645-A
Country	US
Kind code	B2
Filing date	Aug 19, 2022
Priority date	Feb 20, 2020
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Methods, non-transitory computer readable media, and arthroscopic video analysis apparatuses and systems that facilitate improved analysis of videos of arthroscopic procedures are disclosed. With this technology, analytical data related to the video feed of an arthroscopic surgery can be obtained using machine learning models and associated with the video feed. The generated videos can be output in real-time to provide contextual information related to the surgical procedure, or can be saved for playback for training or informational purposes.

First claim

What is claimed is:

1. A method for analyzing surgical video data, the method comprising:
obtaining, by a video data analysis system, video data for a surgical procedure from a camera configured to capture the video data, wherein the video data comprises a field of view of an anatomical region;
generating and providing, by the video data analysis system, a first video data stream and a second video data stream; and
during the providing, by the video data analysis system, of the first video data stream and the second video data stream:
applying, by a video data analysis computing device of the video data analysis system, one or more machine learning models to the second video data stream;
generating, by the video data analysis computing device of the video data analysis system, analytical information related to the anatomical region based on the applied machine learning models; and
associating, by the video data analysis system, the analytical information with the first video data stream.

2. The method of claim 1 , further comprising:
during the providing, by the video data analysis system, of the first video data stream and the second video data stream:
outputting, by the video data analysis computing device of the video data analysis system, one or more overlays containing the analytical information;
correlating, by the video data analysis system, the one or more overlays to locations in the anatomical region related to the analytical information; and
generating, by the video data analysis system, a merged video feed data including the one or more overlays located at the correlated locations in the anatomical region on the first video data stream.

3. The method of claim 2 , further comprising:
during the providing, by the video data analysis system, of the first video data stream and the second video data stream:
outputting, by the video data analysis system, the merged video feed data for display on a display interface, wherein the merged video feed data is output in real-time.

4. The method of claim 1 , wherein the surgical procedure comprises an arthroscopic surgical procedure.

5. The method of claim 1 , wherein the camera is located on an endoscopic device.

6. The method of claim 1 , wherein the analytical information comprises an identification of one or more anatomical structures in the field of view of the anatomical region, an identification of pathologies or defects in the anatomical region, identification of a surgical tool used in the surgical procedure, a location or type of an implant in the anatomical region, a state of the surgical procedure, or one or more geometrical measurements related to anatomical structures in the field of view of the anatomical region.

7. The method of claim 1 , wherein the analytical information comprises a bitmask corresponding to one or more anatomical features in the field of view of the anatomical region.

8. The method of claim 1 , further comprising:
displaying, by the video data analysis system, the first video data stream on a display interface, wherein the first video data stream is displayed without any overlays.

9. The method of claim 1 , wherein generating the first video data stream further comprises, generating, by the video data analysis system, a first high-resolution, low-latency video data stream.

10. The method of claim 1 , further comprising:
downsampling, by the video data analysis system, the second video data stream prior to applying the one or more machine learning models.

11. A video data analysis system comprising:
a camera configured to capture video data;
a video data processing device comprising a first processor coupled to a first memory and configured to execute programmed instructions stored in the first memory comprising:
obtaining video data for a surgical procedure, wherein the video data comprises a field of view of an anatomical region, and
generating and providing a first video data stream and a second video data stream;

and
a video data analysis computing device comprising a second processor coupled to a second memory and configured to execute programmed instructions stored in the second memory comprising:
during the providing, by the first processor, of the first video data stream and the second video data stream:
applying one or more machine learning models to the second video data stream, and
generating analytical information related to the anatomical region based on the applied machine learning models,

wherein the first processor is configured to execute at least one additional programmed instruction stored in the first memory comprising associating the analytical information with the first video data stream.

12. The system of claim 11 , wherein the second processor is configured to execute at least one additional programmed instruction stored in the second memory comprising:
during the providing, by the first processor, of the first video data stream and the second video data stream:
outputting one or more overlays containing the analytical information, and
correlating the one or more overlays to locations in the anatomical region related to the analytical information;

and
wherein the first processor is configured to execute at least one additional programmed instruction stored in the first memory comprising generating merged video feed data including the one or more overlays located at the correlated locations in the anatomical region on the first video data stream.

13. The system of claim 12 , further comprising:
a display interface,
wherein the first processor is configured to execute at least one additional programmed instruction stored in the first memory comprising:
during the providing, by the first processor, of the first video data stream and the second video data stream:
outputting the merged video feed data for display on the display interface,
wherein the merged video feed data is output in real-time.

14. The system of claim 11 , wherein the surgical procedure comprises an arthroscopic surgical procedure.

15. The system of claim 11 , wherein the camera is located on an endoscopic device.

16. The system of claim 11 , wherein the analytical information comprises an identification of one or more anatomical structures in the field of view of the anatomical region, an identification of pathologies or defects in the anatomical region, identification of a surgical tool used in the surgical procedure, a location or type of an implant in the anatomical region, a state of the surgical procedure, or one or more geometrical measurements related to anatomical structures in the field of view of the anatomical region.

17. The system of claim 11 , wherein the analytical information comprises a bitmask corresponding to one or more anatomical features in the field of view of the anatomical region.

18. The system of claim 11 , further comprising:
a display interface,
wherein the first processor is configured to execute at least one additional programmed instruction stored in the first memory comprising displaying the first video data stream on the display interface,
wherein the first video data stream is displayed without any overlays.

19. The system of claim 11 , wherein generating the first video data stream further comprises, generating, by the video data processing device, a first high-resolution, low-latency video data stream.

20. The system of claim 11 , wherein the first processor is configured to execute at least one additional programmed instruction stored in the

Assignees

• Smith & Nephew Inc
• Smith & Nephew Asia Pacific Pte Ltd
• Smith & Nephew Orthopaedics Ag

Inventors

• Quist Brian William
• Jayakumar Vishal
• Rodriguez Carlos A

Classifications

• G06V20/46 Primary

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title

• G16H10/60

  for patient-specific data, e.g. for electronic patient records · CPC title

• G16H30/40

  for processing medical images, e.g. editing · CPC title

• G16H40/63

  for local operation · CPC title