Image processing apparatus, image processing method, and program

Patent metadata

Field	Value
Publication number	US-11766167-B2
Application number	US-202017066504-A
Country	US
Kind code	B2
Filing date	Oct 9, 2020
Priority date	Nov 12, 2014
Publication date	Sep 26, 2023
Grant date	Sep 26, 2023

Abstract

An image processing apparatus includes a special light image acquisition unit that acquires a special light image having information in a specific wavelength band, a generation unit that generates depth information at a predetermined position in a living body using the special light image, and a detection unit that detects a predetermined region using the depth information. The image processing apparatus further includes a normal light image acquisition unit that acquires a normal light image having information in a wavelength band of white light. The specific wavelength band is, for example, infrared light. The generation unit calculates a difference in a depth direction between the special light image and the normal light image to generate depth information at the predetermined position. The detection unit detects a position in which the depth information is a predetermined threshold or more as a bleeding point. The present technology is applicable to an endoscope.

First claim

The invention claimed is:

1. An image processing apparatus for processing an image captured by a medical imaging device, comprising:
circuitry configured to
acquire a special light image that captures a reflected light when light of a specific wavelength band is irradiated to a living body from the medical imaging device,
acquire a white light image that captures a reflected light when white light is irradiated to the living body from the medical imaging device,
generate a first 3D structure image based on the special light image and a second 3D structure image based on the white light image,
compare the first 3D structure image and the second 3D structure image,
determine a difference between the first 3D structure image and the second 3D structure image in a depth direction, and
emphasize an area of an image on a display based based on the difference in depth direction, including at least one of:
emphasize the area by superimposing a numerical value indicating a thickness of a component of the living body from the difference in the depth direction, or
emphasize the area by superimposing a gradation image corresponding to the thickness of the component of the living body from the difference in the depth direction, and, on condition that difference in the depth direction exceeds a predetermined threshold, detect a mist resulting from surgery, emphasize the mist by using an alarm display that indicates presence of the mist or a color that represents the mist an area of an image on a display.

2. The image processing apparatus according to claim 1 , wherein the medical imaging device is an endoscope or a microscope.

3. The image processing apparatus according to claim 1 , wherein the circuitry is further configured to detect a volume, or an overlap of components of the living body in the depth direction based on the difference in the depth direction.

4. The image processing apparatus according to claim 1 , wherein the circuitry is further configured to, on condition that difference in the depth direction exceeds a predetermined threshold, detect a bleeding position.

5. The image processing apparatus according to claim 1 , wherein the circuitry is further configured to instruct display of the special light image or to generate an image via image processing that removes the mist resulting from the surgery.

6. The image processing apparatus according to claim 1 , wherein
the specific wavelength band includes a first wavelength and a second wavelength different from the first wavelength, and
the circuitry is further configured to detect positional information in the depth direction of a blood vessel based on additional depth information generated using a first specific band light image having information limited to the first wavelength and a second specific band light image having information limited to the second wavelength.

7. The image processing apparatus according to claim 6 , wherein the circuitry is further configured to correct an image in which a reflectance of an artery located at a deeper position than a vein using the positional information in the depth direction of the blood vessel.

8. The image processing apparatus according to claim 1 , wherein
the specific wavelength band corresponds to excitation light, and
the circuitry is further configured to detect a positional relationship between overlapping blood vessels or a blood vessel connected to a cancer tissue.

9. The image processing apparatus according to claim 1 , wherein the depth direction is with respect to a direction into the living body or with respect to a direction away from the living body.

10. An image processing method comprising:
acquiring a special light image that captures a reflected light when light of a specific wavelength band is irradiated to a living body from a medical imaging device,
acquiring a white light image that captures a reflected light when white light is irradiated to the living body from the medical imaging device,
generating a first 3D structure image based on the special light image and a second 3D structure image based on the white light image,
comparing the first 3D structure image and the second 3D structure image,
determining a difference between the first 3D structure image and the second 3D structure image in a depth direction, and
emphasizing an area of an image on a display based on the difference in depth direction, including at least one of:
emphasizing the area by superimposing a numerical value indicating a thickness of a component of the living body from the difference in the depth direction, or

emphasizing the area by superimposing a gradation image corresponding to the thickness of the component of the living body from the difference in the depth direction, and
on condition that difference in the depth direction exceeds a predetermined threshold, detecting a mist resulting from surgery, instructing display of the special light image or generating an image via image processing that removes the mist resulting from the surgery.

11. A non-transitory computer readable medium having stored thereon a program that when executed by a computer causes the computer to execute processing, the processing comprising:
acquiring a special light image that captures a reflected light when light of a specific wavelength band is irradiated to a living body from a medical imaging device,
acquiring a white light image that captures a reflected light when white light is irradiated to the living body from the medical imaging device,
generating a first 3D structure image based on the special light image and a second 3D structure image based on the white light image,
comparing the first 3D structure image and the second 3D structure image,
determining a difference between the first 3D structure image and the second 3D structure image in a depth direction, and
emphasizing an area of an image on a display based on the difference in the depth direction, including at least one of:
emphasizing the area by superimposing a numerical value indicating a thickness of a component of the living body from the difference in the depth direction, or
emphasizing the area by superimposing a gradation image corresponding to the thickness of the component of the living body from the difference in the depth direction, and
on condition that difference in the depth direction exceeds a predetermined threshold, detecting a mist resulting from surgery, instructing display of the special light image or generating an image via image processing that removes the mist resulting from the surgery.

12. An image processing apparatus for processing an image captured by a medical imaging device, comprising:
circuitry configured to
acquire a special light image that captures a reflected light when light of a specific wavelength band is irradiated to a living body from the medical imaging device,
acquire a white light image that captures a reflected light when white light is irradiated to the living body from the medical imaging device,
generate a first 3D structure image based on the special light image and a second 3D structure image based on the white light image,
compare the first 3D structure image and the second 3D structure image,
determine a difference between the first 3D structure image and the second 3D structure image in a depth direction, and
on condition that difference in the depth direction exceeds a threshold, detect a mist resulting from surgery, and instruct display of the special light image or generate an image via image processing that removes the mist resulting from the surgery.

Assignees

• Sony Corp

Inventors

• Nakamura Yukihiro
• Mizukura Takami
• Hayashi Tsuneo
• Kikuchi Daisuke

Classifications

• A61B1/0638 Primary

  providing two or more wavelengths · CPC title

• A61B1/00045 Primary

  Display arrangement · CPC title

• A61B1/000094

  extracting biological structures · CPC title

• A61B1/046

  for infrared imaging · CPC title

• A61B5/0086

  using infrared radiation · CPC title