Systems, methods and devices to scan 3D surfaces for intra-operative localization

Patent metadata

Field	Value
Publication number	US-12029505-B2
Application number	US-202318124866-A
Country	US
Kind code	B2
Filing date	Mar 22, 2023
Priority date	Apr 28, 2016
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

Systems and methods are described herein to generate a 3D surface scan of a surface profile of a patient's anatomy. The 3D surface scan may be generated by reflections of structured light off the surface profile of the anatomy. The 3D surface scan may be used during intra-operative surgical navigation by a localization system. Optionally, a pre-operative medical image may also be registered to the localization system or used to enhance the 3D surface scan.

First claim

What is claimed is:

1. A computer program product comprising a non-transient storage device storing instruction that, when executed by one or more processors, cause the one or more processors to perform steps comprising:
receiving sensor data of a surgical site from one or more sensors comprising at least one camera, the surgical site including a patient anatomy, and wherein the patient anatomy is coupled to one or more reference elements, each in a fixed position relative to the patient anatomy, and wherein at least some of the sensor data comprises reference element data for the one or more reference elements and wherein at least some of the sensor data comprises surface profile data and material type data;
determining a spatial surface profile of the surgical site from the sensor data;
determining a type of material at the surgical site and a location of the type of material in the spatial surface profile from the sensor data;
performing a localization registration using the sensor data to register the spatial surface profile and the patient anatomy for surgical localization; and
presenting the spatial surface profile in association with the patient anatomy using the localization registration, including indicating the type of material at the location in the spatial surface profile.

2. The computer program product of claim 1 , wherein:
at least some of the reference element data sensor data comprises scanning extrinsic parameter data received with respect to a scanning reference element of the one or more reference elements;
the spatial surface profile is determined further using the scanning extrinsic parameter data;
at least some of the reference element data comprises registration extrinsic parameter data with respect to a localization reference element of the one or more reference elements, the registration extrinsic parameter data comprising tracking data to calculate a registration position and orientation with respect to the scanning reference element;
the localization system element and the scanning reference element are either a same reference element or different reference elements; and
performing the localization registration comprises registering the spatial surface profile to a localization system using the registration position and orientation of the localization reference element with respect to the scanning reference element to enable intra-operative localization with respect to the spatial surface profile and type of material.

3. The computer program product of claim 1 , wherein the steps further comprise updating the presenting in response to receiving updated sensor data, the updated sensor data providing updated reference element data tracking the patient anatomy.

4. The computer program product of claim 1 , wherein the steps further comprise providing intra-operative navigation with respect to the spatial surface profile of the patient anatomy using the localization registration.

5. The computer program product of claim 4 , wherein the steps further comprise registering a 3D model to the patient anatomy; and wherein the intra-operative navigation with respect to the spatial surface profile and type of material is provided in association with the 3D model.

6. The computer program product of claim 1 , wherein the step of determining the type of material identifies disturbance data using knowledge of an expected surface profile or in response to user input, and the method comprises excluding the disturbance data from the spatial surface profile.

7. The computer program product of claim 1 , wherein the type of material comprises a bone material, a subcutaneous fat material, a cartilage material, a muscle material, a hair material, a tumor material, or a tool material.

8. The computer program product of claim 1 , wherein:
the step of determining the type of material determines a plurality of types of materials, each of the types identified at one or more respective locations; and
the step of presenting comprises indicating each of the types of material at the one or more respective locations.

9. The computer program product of claim 8 , wherein the plurality of types of materials comprises two or more of a bone material, a subcutaneous fat material, a cartilage material, a muscle material, or a tool material.

10. The computer program product of claim 1 , wherein:
the step of determining the type of material determines a plurality of types of materials, each of the types identified at one or more respective locations; and
the step of displaying omits one or more of the types of materials automatically or in response to user selection.

11. The computer program product of claim 1 , wherein the sensor data comprises corresponding i) 3D surface profile data from a camera, and ii) material properties data from a materials sensor.

12. The computer program product of claim 11 , wherein the materials sensor comprises a spectroscopy sensor.

13. The computer program product of claim 12 , wherein the steps further comprise processing spectroscopy information from the material properties data to identify an articular surface of a joint and determine native kinematics of the joint.

14. The computer program product of claim 1 , wherein the sensor data comprises camera data, the camera data providing 3D surface profile data for determining the surface profile and material properties data for identifying the type of material.

15. The computer program product of claim 14 , wherein the camera data is determined from structured light projected by a structure light projector.

16. The computer program product of claim 14 , wherein the camera data comprises multi-spectral data from a multi-spectral camera.

17. The computer program product of claim 1 , wherein the step of determining the type of material comprises correlating at least some of the sensor data to stored reference data for materials.

18. A system comprising:
one or more sensors including at least one camera, the one or more sensors configured to provide sensor data, wherein: at least some of the sensor data comprising 3D surface profile data for determining a spatial surface profile of a surgical site; at least some of the sensor data comprising material properties data for identifying a type of material at the surgical site and at a location in the spatial surface profile; the surgical site comprising a patient anatomy, the patient anatomy coupled to one or more reference elements, each in a fixed position relative to the patient anatomy; and at least some of the sensor data comprising reference element data for the one or more reference elements; and
at least one computing unit each unit comprising one or more processors, the at least one computing unit configured to:
receive the sensor data for the surgical site from the one or more sensors;
determine the spatial surface profile of the surgical site from the sensor data;
determine the type of material and the location of the type of material in the spatial surface profile from the sensor data;
perform a localization registration using the sensor data to register the spatial surface profile and the patient anatomy for surgical localization; and
present the spatial surface profile in association with the patient anatomy using the localization registration, including indicating the type of material at the location in the spatial surface profile.

19. The system of claim 18 , wherein:
the camera is configured to provide the 3D surface profile data and the material properties data as optical data from the camera

Assignees

• Intellijoint Surgical Inc

Inventors

• Hladio Andre Novomir
• Fanson Richard Tyler
• Becker Luke
• Abadpour Arash
• Schipper Joseph Arthur

Classifications

• G06T2207/30016

  Brain · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/337

  involving reference images or patches · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• A61B2090/502

  Headgear, e.g. helmet, spectacles · CPC title