Apparatus for automated panning and digital zooming in robotic surgical systems

Patent metadata

Field	Value
Publication number	US-9699445-B2
Application number	US-201213408236-A
Country	US
Kind code	B2
Filing date	Feb 29, 2012
Priority date	Mar 28, 2008
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

In one embodiment of the invention, a digital zoom and panning system for digital video includes an image acquisition device to capture digital video images; an image buffer to store one or more frames of digital video images as source pixels; a display device having first pixels to display images; a user interface to accept user input including a source rectangle to select source pixels within frames of the digital video images, a destination rectangle to select target pixels within the display device to display images, and a region of interest within the digital video images to display in the destination rectangle; and a digital mapping and filtering device to selectively map and filter source pixels in the region of interest from the image buffer into target pixels of the display device in response to the user input.

First claim

What is claimed is:

1. A digital zoom and panning system for digital video, the system comprising:
an image acquisition device to capture digital video images;
an image buffer coupled to the image acquisition device, the image buffer to store one or more frames of digital video images as source pixels;
a first display device having first pixels to display images;
a first digital mapping and filtering device coupled to the image buffer, the first digital mapping and filtering device to selectively map and filter source pixels in a first source rectangle from the image buffer into target pixels in a first destination rectangle of the first display device; and
a first user interface displayed on the first display device to accept a first user input, including the first source rectangle to select first source pixels within frames of the digital video images and the first destination rectangle to select target pixels within the first display device to display images.

2. The digital zoom and panning system of claim 1 , wherein:
the first digital mapping and filtering device to automatically select a first region of interest within the digital video images to display in the first destination rectangle; and
the first digital mapping and filtering device selectively maps and filters the source pixels in the first region of interest into target pixels of the first display device.

3. The digital zoom and panning system of claim 2 , wherein
the automatic selection of the first region of interest is in response to a tool centroid, a depth of focus, a user focal-point, an average working envelope, or a combination thereof.

4. The digital zoom and panning system of claim 1 , further comprising:
a first user interface to accept a first user input including the first source rectangle to select first source pixels within frames of the digital video images, the first destination rectangle to select target pixels within the first display device to display images, and the first region of interest within the digital video images to display in the first destination rectangle; and
wherein the first digital mapping and filtering device selectively maps and filters the source pixels in the first region of interest into target pixels of the first display device in response to the first user input.

5. The digital zoom and panning system of claim 1 , further comprising:
a first display buffer coupled to and between the first display device and the digital mapping and filtering device, the first display buffer to store one or more frames of target pixels received from the digital mapping and filtering device and write one or more frames of target pixels to the first display device to display images.

6. The digital zoom and panning system of claim 1 , further comprising:
a second display device having target pixels to display images; and
a second digital mapping and filtering device coupled to the image buffer and the user interface, the second digital mapping and filtering device to selectively map and filter source pixels in a second region of interest from the image buffer to digitally zoom into target pixels in a second destination rectangle of the second display device.

7. The digital zoom and panning system of claim 6 , further comprising:
a second user interface to accept second user input including the second source rectangle to select second source pixels within frames of the digital video images, the second destination rectangle to select target pixels within the second display device to display images, and the second region of interest within the digital video images to display in the second destination rectangle.

8. The digital zoom and panning system of claim 1 , further comprising:
a gaze detection system to detect the first region of interest within the digital video images to digitally pan and display into the first destination rectangle in response to a gaze detected from a user's left and right eyes.

9. The digital zoom and panning system of claim 1 , further comprising:
a tool tracking system to detect the first region of interest within the digital video images to digitally pan and display in the first destination rectangle in response to one or more tracked positions of one or more tools by the tool tracking system.

10. The digital zoom and panning system of claim 1 , wherein
the image acquisition device is a robotic endoscopic camera to capture digital video images of a surgical site and to be mechanically panned within the surgical site; and
in response to the detection of the first region of interest being outside the first source rectangle, the robotic endoscopic camera to mechanically pan within the surgical site and the first digital mapping and filtering device to digitally pan and display images in the first region of interest into the first destination rectangle.

11. A digital zoom and panning method for digital video, the method comprising:
capturing digital video images;
storing one or more frames of digital video images as source pixels;
displaying images including a first plurality of pixels on a first display device;
selectively mapping and filtering source pixels in a first source rectangle from the image buffer into target pixels in a first destination rectangle of the first display device; and
accepting a first user input through a first user interface displayed on the first display device, the first user input including a first source rectangle to select first source pixels within frames of the digital video images and the first destination rectangle to select target pixels within the first display device to display images.

12. The digital zoom and panning system method of claim 11 , further comprising:
automatically selecting a first region of interest within the digital video images to display in the first destination rectangle; and
selectively mapping and filtering the source pixels in the first region of interest into target pixels of the first display device.

13. The digital zoom and panning method of claim 12 , wherein
the automatic selecting of the first region of interest is in response to a tool centroid, a depth of focus, a user focal-point, an average working envelope, or a combination thereof.

14. The digital zoom and panning method of claim 11 , wherein the first user input includes a first region of interest within the digital video images to display in the first destination rectangle; and
the method further comprises
selectively mapping and filtering the source pixels in the first region of interest into target pixels of the first display device in response to the first user input.

15. The digital zoom and panning method of claim 11 , further comprising:
storing one or more frames of target pixels to a first display buffer coupled to and between the first display device and a digital mapping filtering device; and
writing one or more frames of target pixels to the first display device.

16. The digital zoom and panning system method of claim 11 , further comprising:
displaying images on a second display device; and
selectively mapping and filter source pixels in a second region of interest from the image buffer to digitally zoom into target pixels in a second destination rectangle of the second display device.

17. The digital zoom and panning method of claim 16 , further comprising:
accepting second user input including the second source rectangle to select second source pixels within frames of the digital video images, the second destination rectangle to select target pixels within the second display dev

Assignees

• Hoffman Brian D
• Hasser Christopher J
• Nowlin William C
• Intuitive Surgical Operations

Inventors

• Hoffman Brian D
• Hasser Christopher J
• Nowlin William C

Classifications

• H04N13/383

  for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes · CPC title

• A61B34/37

  Leader-follower robots (A61B34/35 takes precedence) · CPC title

• A61B2090/371

  with simultaneous use of two cameras · CPC title

• H04N13/296

  Synchronisation thereof; Control thereof · CPC title

• A61B34/30

  Surgical robots · CPC title