Method for real time update of fly-through camera placement

Patent metadata

Field	Value
Publication number	US-12079440-B2
Application number	US-202318138382-A
Country	US
Kind code	B2
Filing date	Apr 24, 2023
Priority date	Oct 22, 2018
Publication date	Sep 3, 2024
Grant date	Sep 3, 2024

Abstract

A virtual endoscopic view shows a surgical area and surrounding anatomy and may also show a position of a surgical instrument in use during a surgical procedure, allowing a surgeon to virtually view the surgical area when direct viewing or actual endoscopic views are incomplete, obstructed, or otherwise unavailable or undesirable. In order to render the endoscopic view, an IGS navigation system may be configured with an observer point and an observer orientation within 3-D space based upon user inputs. A user interface for defining these points allows a user to view a virtual endoscopic preview in real-time while providing inputs, thus improving the likelihood that the resulting virtual endoscopic view is as desired by the user; and reducing time spent redefining and reconfiguring the virtual endoscopic view. The virtual endoscopic preview may provide combinations of static and dynamic images to illustrate the spatial relationship of the provided inputs.

First claim

We claim:

1. A system comprising:
(a) a display;
(b) user input;
(c) a set of preoperative images associated with a patient; and
(d) a processor configured to provide a virtual camera placement interface to a user via the display and receive inputs via the user input; the virtual camera placement interface comprising a set of preoperative image panes and a virtual camera view, each of the set of preoperative image panes comprising a preoperative image from the set of preoperative images, and the user input being operable to move a cursor over and make selections from the set of preoperative image panes, the processor being further configured to:
(i) define a first point based upon a first selection received via the user input, the first selection comprising a point on one of the set of preoperative image panes that corresponds to a first cursor position of the cursor when the first selection is received,
(ii) define a second point based upon a second cursor position of the cursor on any of the set of preoperative image panes,
(iii) display a real-time virtual endoscopic preview in the virtual camera view based upon the first point and the second point,
(iv) change a value of the second point as the cursor is moved based on the user input and the second cursor position changes, and
(v) after display of the real-time virtual endoscopic preview, receive a second selection, the second selection comprising a selected second point determined based upon the second point,
(vi) based upon a modify selection, discarding a selected value of one of the first point or the selected second point,
(vii) defining a modified point based upon a third cursor position, the modified point being one of the first point or the selected second point whose value was discarded, and
(viii) display the real-time virtual endoscopic preview in the virtual camera view based upon a retained point and the modified point, the retained point being one of the first point or the selected second point whose value was not discarded.

2. The system of claim 1 , the processor being further configured to update the real-time virtual endoscopic preview as the second point is changed.

3. The system of claim 1 , the processor being further configured to:
(i) after displaying the real-time virtual endoscopic preview, receive a second selection via the user input, the second selection comprising a selected second point determined based upon the second point, and
(ii) create a virtual camera definition based upon the first point and the selected second point, the virtual camera definition being configured to be usable by an image guided surgery navigation system to produce a virtual endoscopic view during a surgical procedure.

4. The system of claim 3 , the system comprising the image guided surgery navigation system, and the user input comprising a pointing device.

5. The system of claim 1 , the processor being further configured to, while displaying the real-time virtual endoscopic preview:
(i) determine a spatial relationship between the first point and the second point,
(ii) create a route through the set of preoperative images based upon the spatial relationship, and
(iii) display a sequence of flythrough images in the virtual camera view based upon the route.

6. The system of claim 5 , the sequence of flythrough images comprising a plurality of sequential images selected from the set of preoperative images and arranged in a determined order that the plurality of sequential images would be viewed while traversing the route.

7. The system of claim 5 , the virtual camera placement interface comprising a set of navigation controls, and the processor being further configured to, in response to inputs via the set of navigation controls, adjust the speed and a determined order at which the sequence of flythrough images is displayed.

8. The system of claim 7 , the processor being further configured to, in response to inputs via the set of navigations controls:
(i) pause display of the sequence of flythrough images,
(ii) display a new image from the set of preoperative images in the virtual camera view based upon inputs indicating a change to one or both of a view position and orientation from an initial view position and orientation provided by the route, and
(iii) change a value of one or both of the first point and the second point based upon the new image.

9. The system of claim 8 , the processor being further configured to, while the new image is displayed, update one or more of the preoperative image panes to comprise a new preoperative image from the set of preoperative images, the new preoperative image for each being determined based upon a relationship of the new preoperative image to the new image.

10. The system of claim 5 , the first point being associated with a location of a virtual camera and the second point is associated with an orientation of the virtual camera, and the route comprising a start point that is determined based upon the first point and an end point that is determined based upon the second point.

11. The system of claim 5 , the second point being associated with a location of a virtual camera and the first point is associated with an orientation of the virtual camera, and the route comprising a start point based that is determined based upon the second point and an end point that is determined based upon the first point.

12. The system of claim 1 , the processor being further configured to:
(i) after displaying the real-time virtual endoscopic preview, receive a second selection via the user input, the second selection comprising a selected second point determined based upon the second point,
(ii) based upon a swap selection received via the user input, swap values of the first point and the selected second point, and
(iii) display the real-time virtual endoscopic preview in the virtual camera view based upon the swapped values of the first point and the selected second point.

13. The system of claim 1 , the processor being further configured to, prior to the first selection being received via the user input:
(i) define the first point based upon another first cursor position on one of the set of preoperative image panes, that preoperative image pane being associated with a preoperative image of the set of preoperative images, and the preoperative image comprising a perspective indicating an orientation from which the preoperative image is viewed,
(ii) define the second point based upon the perspective of the preoperative image, and
(iii) display the real-time virtual endoscopic preview in the virtual camera view based upon the first point and the second point.

14. A method for configuring a virtual camera in 3-D space with a virtual camera placement interface, comprising the steps:
(a) displaying a set of preoperative image panes via the virtual camera placement interface, each of the set of preoperative image panes comprising a preoperative image from a set of preoperative images associated with a patient;
(b) defining a first point based upon receiving a first selection from a user via the virtual camera placement interface, the first selection comprising a point on one of the set of preoperative image panes that corresponds to a first cursor position of a cursor positioned by the user over that preoperative image pane;
(c) defining a second point based upon a second cursor position of the cursor on any of the set of preoperative image panes;
(d) displaying a real-time virtual endoscopic preview via the virtual camera placement interface based upon the first point and the second point; and
(e) after displaying the real

Assignees

• Acclarent Inc
• Biosense Webster Israel Ltd

Inventors

• Palushi Jetmir
• Salazar Henry F
• Trott Jordan R
• Levi Moran
• Bustan Itamar
• Pinsky Yoav
• Racheli Noam
• Papadakis Athanasios

Classifications

• A61B34/25

  User interfaces for surgical systems · CPC title

• A61B2090/3762

  using computed tomography systems [CT] · CPC title

• A61B2090/374

  NMR or MRI · CPC title

• A61B2034/105

  Modelling of the patient, e.g. for ligaments or bones · CPC title

• G06F2203/04803

  Split screen, i.e. subdividing the display area or the window area into separate subareas · CPC title