Image processing device, image processing method, and program

Patent metadata

Field	Value
Publication number	US-2016203602-A1
Application number	US-201414914780-A
Country	US
Kind code	A1
Filing date	Jul 24, 2014
Priority date	Sep 10, 2013
Publication date	Jul 14, 2016
Grant date	—

Abstract

[Object] To improve work efficiency with respect to work using a transparent object, such as a transparent operation tool, for example.
[Solution] An image region where an object exists as a target region is detected on the basis of a second captured image, when a first captured image is a captured image obtained by selectively receiving a light of a first wavelength band, and the second captured image is a captured image obtained by selectively receiving a light of a second wavelength band, the captured images being obtained by capturing the object that is transparent for the light of the first wavelength band and is opaque for the light of the second wavelength band. Subsequently, an outline of the object is superimposed on the first captured image on the basis of information of the target region detected by the target detecting unit.

First claim

1 . An image processing device comprising:
a target detecting unit configured to detect an image region where an object exists as a target region on the basis of a second captured image, when a first captured image is a captured image obtained by selectively receiving a light of a first wavelength band, and the second captured image is a captured image obtained by selectively receiving a light of a second wavelength band, the captured images being obtained by capturing the object that is transparent for the light of the first wavelength band and is opaque for the light of the second wavelength band; and an outline superimposing unit configured to superimpose an outline of the object on the first captured image on the basis of information of the target region detected by the target detecting unit.

2 . The image processing device according to claim 1 , wherein
the target detecting unit detects the target region on the basis of the second captured image and the first captured image.

3 . The image processing device according to claim 1 , wherein
the target detecting unit
generates a second edge image based on the second captured image and a first edge image based on the first captured image by performing an edge extraction with respect to each of the second captured image and the first captured image,
performs a template matching using a template of the object for each of the second edge image and the first edge image, and
detects, as the target region, an image region that is detected by the template matching for the second edge image and does not overlap with an image region detected by the template matching for the first edge image.

4 . The image processing device according to claim 3 , wherein
the target detecting unit
performs the edge extraction with respect to the second captured image and a plurality of first captured images obtained by selectively receiving lights of different wavelength bands in the first wavelength band, and generates the second edge image based on the second captured image and a plurality of first edge images based on the plurality of first captured images,
performs the template matching for each of the generated edge images, and
detects, as the target region, an image region that is detected by the template matching for the second edge image and does not overlap with any one of image regions detected by the template matching for the plurality of first edge images.

5 . The image processing device according to claim 1 , wherein,
when a left eye side second captured image and a right eye side second captured image are a left eye image and a right eye image obtained by stereoscopically capturing images of the object by means of an image sensor that selectively receives a light of the second wavelength band, respectively, the target detecting unit detects the target region on the basis of distance information calculated on the basis of a parallax between the left eye side second captured image and the right eye side second captured image.

6 . The image processing device according to claim 5 , wherein
the target detecting unit generates a distance image of the second captured image and a distance image of the first captured image on the basis of the distance information calculated with respect to each of the second captured image and the first captured image, generates a difference image between the distance image of the second captured image and the distance image of the first captured image, and detects the target region on the basis of the difference image.

7 . The image processing device according to claim 1 , wherein
the target detecting unit decides the target region used in superimposition of the outline by the outline superimposing unit, on the basis of a detection result of the target region with respect to a plurality of frame images.

8 . The image processing device according to claim 1 , comprising:
an optical distortion correcting unit configured to correct an optical distortion generated by the object with respect to the target region in the first captured image.

9 . The image processing device according to claim 8 , wherein
the target detecting unit identifies a type of the object on the basis of the second captured image, and the optical distortion correcting unit corrects the optical distortion with a correction characteristic according to the type of the object identified by the target detecting unit.

10 . The image processing device according to claim 8 , wherein,
when a left eye side second captured image and a right eye side second captured image are a left eye image and a right eye image obtained by stereoscopically capturing images of the object by means of an image sensor that selectively receives a light of the second wavelength band, respectively, the target detecting unit generates three-dimensional position information of the object on the basis of the left eye side second captured image and the right eye side second captured image, and detects an orientation of the object on the basis of the three-dimensional position information, and the optical distortion correcting unit corrects the optical distortion with a correction characteristic according to the orientation of the object detected by the target detecting unit.

11 . The image processing device according to claim 10 , wherein
the optical distortion correcting unit corrects the optical distortion with a correction characteristic according to a distance of the object.

12 . An image processing method comprising:
a target detection procedure for detecting, as a target region, an image region where an object exists on the basis of a second captured image, when a first captured image is a captured image obtained by selectively receiving a light of a first wavelength band, and the second captured image is a captured image obtained by selectively receiving a light of a second wavelength band, the captured images being obtained by capturing the object that is transparent for the light of the first wavelength band and is opaque for the light of the second wavelength band; and an outline superimposition procedure for superimposing an outline of the object on the first captured image on the basis of information of the target region detected by the target detection procedure.

13 . A program for causing a signal processing device to execute:
a target detection process for detecting, as a target region, an image region where an object exists on the basis of a second captured image, when a first captured image is a captured image obtained by selectively receiving a light of a first wavelength band, and the second captured image is a captured image obtained by selectively receiving a light of a second wavelength band, the captured images being obtained by capturing the object that is transparent for the light of the first wavelength band and is opaque for the light of the second wavelength band; and an outline superimposition process for superimposing an outline of the object on the first captured image on the basis of information of the target region detected by the target detection process.

Assignees

• Sony Corp

Inventors

• Hayashi Tsuneo

Classifications

• H04N23/555

  for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes · CPC title

• G06V10/46

  Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56) · CPC title

• G06V10/143

  Sensing or illuminating at different wavelengths · CPC title

• H04N13/239 Primary

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• A61B1/00194

  adapted for three-dimensional imaging · CPC title