Devices, systems and methods for natural feature tracking of surgical tools and other objects

Patent metadata

Field	Value
Publication number	US-12588953-B2
Application number	US-202218082016-A
Country	US
Kind code	B2
Filing date	Dec 15, 2022
Priority date	Oct 29, 2014
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

Systems, methods and devices for use in tracking are described, using optical modalities to detect spatial attributes or natural features of objects, such as, tools and patient anatomy. Spatial attributes or natural features may be known or may be detected by the tracking system. The system, methods and devices can further be used to verify a calibration of a tool either by a computing unit or by a user. Further, the disclosure relates to detection of spatial attributes, including depth information, of the anatomy for purposes of registration or to create a 3D surface profile of the anatomy.

First claim

What is claimed is:

1 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the processing unit, cause the system to perform a method comprising:
storing first natural feature information of a first object, the natural feature information defining information to identify the first object in the images using natural visual features of the first object;
receiving from the sensor unit an image of the first object;
extracting second natural feature information from natural visual features of the first object in the image; and
calculating the pose of the first object relative to the sensor unit using the first natural feature information and the second natural feature information to facilitate performance of the surgical navigation procedure;
wherein each of the first natural feature information and second natural feature information is defined from one or more of the natural visual features comprising inherent spatial features comprising any of a shape feature, a pattern feature, a configuration feature or an ornamentation feature; and wherein the one or more of the natural visual features relate to portions of the first object without including any markers or identifiers applied to the first object for purposes of tracking the first object.

2 . The system of claim 1 , wherein the sensor unit is configured for attachment to a portion of a patient's anatomy.

3 . The system of claim 2 , wherein the portion is a pelvic bone.

4 . The system of claim 1 , wherein the first object is an acetabular impactor.

5 . The system of claim 1 , wherein the instructions cause the system to simultaneously localize targets and natural visual features from the image received from the sensor unit.

6 . The system of claim 5 , wherein:
the image received includes the first object and a second object, the second object having a target coupled thereto; and the instructions cause the system to:
localize the first object via the first natural feature information and the second natural feature information; and
localize the second object from target information obtained from the image and stored target information for the target.

7 . The system of claim 6 , wherein the first object comprises a first instrument and the second object a second instrument and wherein the instructions cause the system to determine poses of each of the first instrument and second instrument.

8 . The system of claim 6 , wherein:
the first object is an acetabular impactor; the second object is a patient leg; and the instructions cause the system to:
localize the acetabular impactor relative to a patient pelvis; and
localize a position of the patient leg relative to the patient pelvis using a target attached to the leg.

9 . The system of claim 1 , wherein the instructions cause the system to obtain the first natural feature information from a digital file accessible to the processing unit.

10 . The system of claim 9 , wherein the first object is a tool comprising an effector, and the first natural feature information includes a positional relationship of the effector with respect to other parts of the tool.

11 . The system of claim 1 , wherein to calculate the pose the instructions cause the system to perform optimization operations to: match at least some of the first natural feature information and at least some of the second natural feature information; and calculate the pose responsive to the match.

12 . The system of claim 1 , wherein the instructions cause the system to compute the first natural feature information using a feature learning procedure in which sufficient views of the object are presented to the processing unit for learning its natural visual features.

13 . The system of claim 1 , wherein the sensor is configured with a light to illuminate the natural visual features in the field of view.

14 . The system of claim 1 , wherein the first natural feature information is derived from a medical image of the patient comprising a Computed Tomography (CT) scan, a Magnetic Resonance Image (MRI) scan, or another pre-operative scan of the patient.

15 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the processing unit, cause the system to perform a method comprising:
storing first natural feature information of a first object, the natural feature information defining information to identify the first object in the images using natural visual features of the first object;
receiving from the sensor unit an image of the first object;
extracting second natural feature information from natural visual features of the first object in the image; and
calculating the pose of the first object relative to the sensor unit using the first natural feature information and the second natural feature information to facilitate performance of the surgical navigation procedure;
wherein the pose is calculated in any number of degrees of freedom between 2 degrees of freedom and 6 degrees of freedom.

16 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the processing unit, cause the system to perform a method comprising:
storing first natural feature information of a first object, the natural feature information defining information to identify the first object in the images using natural visual features of the first object;
receiving from the sensor unit an image of the first object;
extracting second natural feature information from natural visual features of the first object in the image; and
calculating the pose of the first object relative to the sensor unit using the first natural feature information and the second natural feature information to facilitate performance of the surgical navigation procedure;
wherein the first natural feature information and the second natural feature information each comprises information defining edges of a cylindrical shaft of the first object.

17 . The system of claim 16 , wherein operations to calculate the pose account for a perspective effect of the natural visual features in the image.

18 . The system of claim 16 , wherein the first natural feature information is derived from a medical image of the patient comprising a Computed Tomography (CT) scan, a Magnetic Resonance Image (MRI) scan, or another pre-operative scan of the patient.

19 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the proces

Assignees

• Intellijoint Surgical Inc

Inventors

• Hladio Andre Novomir
• Bakirtzian Armen Garo
• Fanson Richard Tyler

Classifications

• G06T2207/30008

  Bone · CPC title

• G06T7/337

  involving reference images or patches · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• A61B90/06

  Measuring instruments not otherwise provided for · CPC title

• A61B2034/105

  Modelling of the patient, e.g. for ligaments or bones · CPC title