System and method for specular reflection detection and reduction

Patent metadata

Field	Value
Publication number	US-11937899-B2
Application number	US-201916562121-A
Country	US
Kind code	B2
Filing date	Sep 5, 2019
Priority date	Feb 27, 2014
Publication date	Mar 26, 2024
Grant date	Mar 26, 2024

Abstract

An imaging system includes an illuminator for providing illumination on a region of interest, one or more first detectors for detecting reflections of the illumination, and a processing unit. The processing unit is configured to: receive a first image containing the region of interest; determine a first region of high specular reflection in the first image based on the reflections of the illumination on the region of interest; mask the first image by substituting a default color designating masking to alert a user to incomplete image information in the first region of high specular reflection to create a masked first image; and generate a composite image based on the masked first image and a fluorescence image of the region of interest.

First claim

What is claimed is:

1. An imaging system comprising:
an illuminator for providing illumination on a region of interest;
one or more first detectors for detecting reflections of the illumination; and
a processing unit configured to:
receive a first image containing the region of interest;
determine a first region of high specular reflection in the first image based on the reflections of the illumination on the region of interest;
mask the first image by substituting a default color designating masking to alert a user to incomplete image information in the first region of high specular reflection to create a masked first image; and
generate a composite image based on the masked first image and a fluorescence image of the region of interest.

2. The imaging system of claim 1 , wherein the processing unit is further configured to automatically add one or more visual highlights associated with the first region of high specular reflection to the composite image.

3. The imaging system of claim 1 , further comprising:
one or more second illuminators for triggering fluorescing of one or more fluorescent materials in the region of interest; and
one or more second detectors for detecting the fluorescing of the fluorescent materials;
wherein the processing unit is further configured to generate the fluorescence image of the region of interest.

4. The imaging system of claim 1 , wherein the processing unit is further configured to receive a second image from the one or more first detectors, the second image including the detected reflections and being based on a second imaging geometry different from a first imaging geometry of the first image.

5. The imaging system of claim 4 , wherein the processing unit is further configured to:
estimate a first surface normal in the first region of high specular reflection based on differences between the first and second imaging geometries of the first image and the second image; and
provide one or more move away hints based on the estimated first surface normal.

6. The imaging system of claim 5 , wherein at least one of the move away hints comprises providing a move away line to a motion planning system, the move away line being collinear with the first surface normal.

7. The imaging system of claim 5 , wherein the processing unit is further configured to:
move an imaging unit of the imaging system from a first pose to a second pose, the move altering the first imaging geometry to a third imaging geometry and the second imaging geometry to a fourth imaging geometry;
receive a third image from the one or more first detectors, the third image including the detected reflections and being based on the third imaging geometry;
receive a fourth image from the one or more first detectors, the fourth image including the detected reflections and being based on the fourth imaging geometry;
determine a second region of high specular reflection in the third image and the fourth image;
estimate a second surface normal in the second region of high specular reflection based on differences between imaging geometries of the third image and the fourth image; and
select for use either the first and second images or the third and fourth images based on the first and second surface normals, the first and second regions of high specular reflection, and the first and second poses.

8. The imaging system of claim 5 , wherein the at least one of the move away hints comprise providing a move away point to a motion planning system, the move away point being pointed to by the first surface normal.

9. The imaging system of claim 8 , wherein the move away point is located at a height above the first surface normal based on a working distance of an imaging unit of the imaging system.

10. The imaging system of claim 8 , wherein the move away point is located near an imaging unit of the imaging system.

11. A method comprising:
providing, via an illuminator, illumination on a region of interest;
detecting, via one or more detectors, reflections of the illumination;
receiving, via a processing unit, a first image containing the region of interest;
determining a first region of high specular reflection in the first image based on the reflections of the illumination on the region of interest;
masking the first image by substituting a default color designating masking to alert a user to incomplete image information in the first region of high specular reflection to create a masked first image; and
generating a composite image based on the masked first image and a fluorescence image of the region of interest.

12. The method of claim 11 , further comprising:
automatically adding, via the processing unit, one or more visual highlights associated with the first region of high specular reflection to the composite image.

13. The method of claim 11 , further comprising:
triggering fluorescing of one or more fluorescent materials in the region of interest;
detecting the fluorescing of the fluorescent materials; and
generating the fluorescence image of the region of interest.

14. The method of claim 11 , further comprising:
receiving a second image from the one or more detectors, the second image including the detected reflections and being based on a second imaging geometry different from a first imaging geometry of the first image.

15. The method of claim 14 , further comprising:
estimating a first surface normal in the first region of high specular reflection based on differences between the first and second imaging geometries of the first image and the second image; and
providing one or more move away hints based on the estimated first surface normal.

16. The method of claim 15 , wherein at least one of the move away hints comprises providing a move away line to a motion planning system, the move away line being collinear with the first surface normal.

17. The method of claim 15 , further comprising:
moving an imaging unit from a first pose to a second pose, the move altering the first imaging geometry to a third imaging geometry and the second imaging geometry to a fourth imaging geometry;
receiving a third image from the one or more detectors, the third image including the detected reflections and being based on the third imaging geometry;
receiving a fourth image from the one or more detectors, the fourth image including the detected reflections and being based on the fourth imaging geometry;
determining a second region of high specular reflection in the third image and the fourth image;
estimating a second surface normal in the second region of high specular reflection based on differences between imaging geometries of the third image and the fourth image; and
selecting for use either the first and second images or the third and fourth images based on the first and second surface normals, the first and second regions of high specular reflection, and the first and second poses.

18. The method of claim 15 , wherein the at least one of the move away hints comprises providing a move away point to a motion planning system, the move away point being pointed to by the first surface normal.

19. The method of claim 18 , wherein the move away point is located at a height above the first surface normal based on a working distance of an imaging unit associated with the one or more detectors.

20. The method of claim 18 , wherein the move away point is located near an imaging unit associated with the one or more detectors.

Assignees

• Intuitive Surgical Operations

Inventors

• Azizian Mahdi
• Sorger Jonathan M

Classifications

• A61B5/0071 Primary

  by measuring fluorescence emission · CPC title

• A61B5/0033 Primary

  Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room · CPC title

• A61B5/7405

  using sound · CPC title

• A61B5/742

  using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339) · CPC title

• A61B2576/00

  Medical imaging apparatus involving image processing or analysis (A61B1/00009, A61B6/52 and A61B8/52 take precedence) · CPC title