System and method for specular reflection detection and reduction

Patent metadata

Field	Value
Publication number	US-12376749-B2
Application number	US-202418435905-A
Country	US
Kind code	B2
Filing date	Feb 7, 2024
Priority date	Feb 27, 2014
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

A system and method of specular reflection detection and reduction includes a processing unit including one or more processors and an imaging unit coupled to the processing unit. The imaging unit includes one or more first illuminators for providing illumination of a region of interest, one or more first detectors for detecting reflections of the illumination, one or more second illuminators for triggering fluorescing of one or more fluorescent materials in the region of interest, and one or more second detectors for detecting the fluorescing of the fluorescent materials. The processing unit is configured to receive a first image from the first detectors, determine one or more regions of high specular reflection in the first image, mask out the regions of high specular reflection in the first image, and generate a composite image based on the masked first image and the detected fluorescence. The first image includes the detected reflections.

First claim

What is claimed is:

1. An imaging system comprising:
a processing unit comprising one or more processors; and
an imaging unit coupled to the processing unit and comprising:
one or more first illuminators providing illumination of a region of interest; and
one or more first detectors for detecting reflections of the illumination;

wherein the processing unit is configured to:
receive a first image from the one or more first detectors, the first image including the detected reflections and being based on a first imaging geometry;
receive a second image from the one or more first detectors, the second image including the detected reflections and being based on a second imaging geometry different from the first imaging geometry;
determine a first region of high specular reflection in the first image and the second image;
estimate a first surface normal in the first region of high specular reflection based on differences between the first imaging geometry and the second imaging geometry; and
provide a move away hint based on the estimated first surface normal, the move away hint indicating a direction of movement of the imaging unit to reduce specular reflection.

2. The imaging system of claim 1 , wherein providing the move away hint comprises providing a move away line to a motion planning system, the move away line being collinear with the first surface normal.

3. The imaging system of claim 1 , wherein the processing unit is further configured to:
move the imaging unit from a first pose to a second pose, the move altering the first imaging geometry to a third imaging geometry and altering the second imaging geometry to a fourth imaging geometry;
receive a third image from the one or more first detectors, the third image including the detected reflections and being based on the third imaging geometry;
receive a fourth image from the one or more first detectors, the fourth image including the detected reflections and being based on the fourth imaging geometry;
determine a second region of high specular reflection in the third image and the fourth image;
estimate a second surface normal in the second region of high specular reflection based on differences between the third imaging geometry and the fourth imaging geometry; and
select for use either the first and second images or the third and fourth images based on the first and second surface normals, the first and second regions of high specular reflection, and the first and second poses.

4. The imaging system of claim 1 , wherein:
the imaging unit further comprises:
one or more second illuminators for triggering fluorescing of one or more fluorescent materials in the region of interest; and
one or more second detectors for detecting the fluorescing of the fluorescent materials; and

the processing unit is further configured to:
mask out the first region of high specular reflection in the first image to create a masked first image;
generate a fluorescence image of the region of interest; and
generate a composite image based on the masked first image and the fluorescence image.

5. The imaging system of claim 1 , wherein the move away hint is based on one or more kinematic models of an arm supporting the imaging unit.

6. The imaging system of claim 1 , wherein the move away hint comprises a visual hint displayed to an operator indicating a suggested direction of movement of the imaging unit.

7. The imaging system of claim 1 , wherein the move away hint comprises a visual hint displayed to an operator indicating a suggested change in orientation of the imaging unit.

8. The imaging system of claim 1 , wherein providing the move away hint comprises providing a move away point to a motion planning system, the move away point being pointed to by the first surface normal.

9. The imaging system of claim 8 , wherein the move away point is located at a height above the first surface normal based on a working distance of the imaging unit.

10. The imaging system of claim 8 , wherein the move away point is located near the imaging unit.

11. A method comprising:
receiving a first image from one or more first detectors of an imaging unit configured to detect reflections of illumination of a region of interest provided by one or more first illuminators of the imaging unit, the first image including the detected reflections and being based on a first imaging geometry;
receiving a second image from the one or more first detectors, the second image including the detected reflections and being based on a second imaging geometry different from the first imaging geometry;
determining a first region of high specular reflection in the first image and the second image;
estimating a first surface normal in the first region of high specular reflection based on differences between the first imaging geometry and the second imaging geometry; and
providing a move away hint based on the first surface normal, the move away hint indicating a direction of movement of the imaging unit to reduce specular reflection.

12. The method of claim 11 , wherein providing the move away hint comprises providing a move away line to a motion planning system, the move away line being collinear with the first surface normal.

13. The method of claim 11 , further comprising:
moving the imaging unit from a first pose to a second pose, the move altering the first imaging geometry to a third imaging geometry and altering the second imaging geometry to a fourth imaging geometry;
receiving a third image from the one or more first detectors, the third image including the detected reflections and being based on the third imaging geometry;
receiving a fourth image from the one or more first detectors, the fourth image including the detected reflections and being based on the fourth imaging geometry;
determining a second region of high specular reflection in the third image and the fourth image;
estimating a second surface normal in the second region of high specular reflection based on differences between the third imaging geometry and the fourth imaging geometry; and
selecting for use either the first and second images or the third and fourth images based on the first and second surface normals, the first and second regions of high specular reflection, and the first and second poses.

14. The method of claim 11 , wherein the move away hint is based on one or more kinematic models of an arm supporting the imaging unit.

15. The method of claim 11 , wherein the move away hint comprises a visual hint displayed to an operator indicating a suggested direction of movement of the imaging unit.

16. The method of claim 11 , wherein the move away hint comprises a visual hint displayed to an operator indicating a suggested change in orientation of the imaging unit.

17. The method of claim 11 , wherein providing the move away hint comprises providing a move away point to a motion planning system, the move away point being pointed to by the first surface normal.

18. The method of claim 17 , wherein the move away point is located at a height above the first surface normal based on a working distance of the imaging unit.

19. The method of claim 17 , wherein the move away point is located near the imaging unit.

20. A non-transitory computer-readable medium comprising a plurality of computer-readable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to:
receive a first image from one or more first detectors of an imaging unit configured to detect

Assignees

• Intuitive Surgical Operations

Inventors

• Azizian Mahdi
• Sorger Jonathan

Classifications

• A61B2576/00

  Medical imaging apparatus involving image processing or analysis (A61B1/00009, A61B6/52 and A61B8/52 take precedence) · CPC title

• A61B5/0033 Primary

  Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room · CPC title

• A61B5/742

  using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339) · CPC title

• A61B5/7405

  using sound · CPC title

• A61B5/0071 Primary

  by measuring fluorescence emission · CPC title