Image processing device, endoscope apparatus, information storage device, and image processing method

Patent metadata

Field	Value
Publication number	US-2016014328-A1
Application number	US-201514861209-A
Country	US
Kind code	A1
Filing date	Sep 22, 2015
Priority date	Mar 27, 2013
Publication date	Jan 14, 2016
Grant date	—

Abstract

An image processing device includes an image acquisition section that acquires a captured image that includes an image of the object, a distance information acquisition section that acquires distance information based on the distance from an imaging section to the object when the imaging section captured the captured image, an in-focus determination section that determines whether or not the object is in focus within a pixel or an area within the captured image based on the distance information, a classification section that performs a classification process that classifies the structure of the object, and controls the target of the classification process corresponding to the results of the determination as to whether or not the object is in focus within the pixel or the area, and an enhancement processing section that performs an enhancement process on the captured image based on the results of the classification process.

First claim

What is claimed is:

1 . An image processing device comprising:
an image acquisition section that acquires a captured image that includes an image of an object; a distance information acquisition section that acquires distance information based on a distance from an imaging section to the object when the imaging section captured the captured image; an in-focus determination section that determines whether or not the object is in focus within a pixel or an area within the captured image based on the distance information; a classification section that performs a classification process that classifies a structure of the object, and controls a target of the classification process corresponding to results of the determination as to whether or not the object is in focus within the pixel or the area; and an enhancement processing section that performs an enhancement process on the captured image based on results of the classification process.

2 . The image processing device as defined in claim 1 ,
the classification section outputting a classification result that corresponds to an out-of-focus state with respect to the pixel or the area for which it has been determined that the object is out of focus.

3 . The image processing device as defined in claim 2 ,
the classification section correcting the result of the classification process to a classification that corresponds to the out-of-focus state with respect to the pixel or the area for which it has been determined that the object is out of focus.

4 . The image processing device as defined in claim 3 ,
the classification section determining whether or not the pixel or the area agrees with characteristics of a normal structure to classify the pixel or the area as a normal part or a non-normal part, and correcting a classification result that represents the normal part or the non-normal part to an unknown state with respect to the pixel or the area for which it has been determined that the object is out of focus, the unknown state representing that it is unknown whether the pixel or the area should be classified as the normal part or the non-normal part.

5 . The image processing device as defined in claim 2 ,
the classification section excluding the pixel or the area for which it has been determined that the object is out of focus from the target of the classification process, and classifying the pixel or the area as a classification that corresponds to the out-of-focus state.

6 . The image processing device as defined in claim 5 ,
the classification section determining whether or not the pixel or the area agrees with characteristics of a normal structure to classify the pixel or the area as a normal part or a non-normal part, excluding the pixel or the area for which it has been determined that the object is out of focus from the target of the classification process that classifies the pixel or the area as the normal part or the non-normal part, and classifying the pixel or the area as an unknown state that represents that it is unknown whether the pixel or the area should be classified as the normal part or the non-normal part.

7 . The image processing device as defined in claim 1 , further comprising:
a depth-of-field acquisition section that acquires depth-of-field information about the imaging section; and a comparison section that compares the distance information with the depth-of-field information, the in-focus determination section determining whether or not the object is in focus within the pixel or the area based on a comparison result of the comparison section.

8 . The image processing device as defined in claim 7 ,
the in-focus determination section determining that the object is in focus within the pixel or the area when the comparison result represents that the distance to the object within the pixel or the area that is represented by the distance information is within a depth of field that is represented by the depth-of-field information.

9 . The image processing device as defined in claim 7 , further comprising:
a focus control section that controls a position of a focus lens that is included in the imaging section, the depth-of-field acquisition section acquiring the depth-of-field information that corresponds to the position of the focus lens.

10 . The image processing device as defined in claim 9 , further comprising:
a control section that controls a position of a zoom lens that is included in the imaging section, the depth-of-field acquisition section acquiring the depth-of-field information that corresponds to a combination of the position of the zoom lens and the position of the focus lens.

11 . The image processing device as defined in claim 7 , further comprising:
a control section that controls a position of a zoom lens that is included in the imaging section, the depth-of-field acquisition section acquiring the depth-of-field information that corresponds to the position of the zoom lens.

12 . The image processing device as defined in claim 1 , further comprising:
an AF control section that controls an autofocus operation that is performed by the imaging section, the in-focus determination section determining whether or not the object is in focus in each of a plurality of frames in which the autofocus operation is performed, and the classification section outputting the result of the classification process that corresponds to a frame among the plurality of frames in which it has been determined that the object is in focus as a final classification result with respect to the pixel or the area for which it has been determined that the object is in focus in the frame among the plurality of frames.

13 . The image processing device as defined in claim 2 ,
the enhancement processing section enhancing the pixel or the area for which the classification section has output the classification result that corresponds to the out-of-focus state.

14 . The image processing device as defined in claim 13 ,
the classification section determining whether or not the pixel or the area agrees with characteristics of a normal structure to classify the pixel or the area as a normal part or a non-normal part, and outputting an unknown state as the classification result that corresponds to the out-of-focus state, the unknown state representing that it is unknown whether the pixel or the area should be classified as the normal part or the non-normal part, and the enhancement processing section enhancing the pixel or the area that has been classified as the unknown state by the classification section.

15 . The image processing device as defined in claim 1 ,
the classification section determining whether or not the pixel or the area agrees with characteristics of a normal structure to classify the pixel or the area as a normal part or a non-normal part, and the enhancement processing section enhancing the pixel or the area that has been classified as the non-normal part by the classification section.

16 . The image processing device as defined in claim 1 ,
the classification section performing the classification process that classifies the structure of the object based on the distance information.

17 . The image processing device as defined in claim 16 , further comprising:
a known characteristic information acquisition section that acquires known characteristic information, the known characteristic information being information that represents known characteristics relating to a structure of the object, the classification section including: a surface shape

Assignees

• Olympus Corp

Inventors

• Rokutanda Etsuko

Classifications

• G06F18/24

  Classification techniques · CPC title

• A61B1/000095 Primary

  for image enhancement · CPC title

• G06K9/6267

  Physics · mapped topic

• A61B1/05

  characterised by the image sensor, e.g. camera, being in the distal end portion · CPC title

• G06K9/52

  Physics · mapped topic