Image processing apparatus, image processing method, and non-transitory computer-readable medium

Patent metadata

Field	Value
Publication number	US-2016155235-A1
Application number	US-201514949136-A
Country	US
Kind code	A1
Filing date	Nov 23, 2015
Priority date	Nov 28, 2014
Publication date	Jun 2, 2016
Grant date	—

Abstract

There is provided with an image processing apparatus. A captured image of a target object that is captured by an image capturing apparatus is obtained. Information that indicates a deterioration degree of the captured image is obtained for a position in the captured image. A feature of the target object is extracted from the captured image based on the deterioration degree. The feature of the target object and a feature of the three-dimensional model observed when the three-dimensional model is arranged in accordance with a predetermined position and orientation are associated. A position and orientation of the target object with respect to the image capturing apparatus are derived by correcting the predetermined position and orientation based on a result of association.

First claim

What is claimed is:

1 . An image processing apparatus comprising:
an image obtaining unit configured to obtain a captured image of a target object that is captured by an image capturing apparatus; a deterioration degree obtaining unit configured to obtain information that indicates a deterioration degree of the captured image, for a position in the captured image; an extraction unit configured to extract a feature of the target object from the captured image based on the deterioration degree; a model holding unit configured to hold a three-dimensional model of the target object; an associating unit configured to associate the feature of the target object and a feature of the three-dimensional model observed when the three-dimensional model is arranged in accordance with a predetermined position and orientation; and a deriving unit configured to derive a position and orientation of the target object with respect to the image capturing apparatus by correcting the predetermined position and orientation based on a result of association.

2 . The image processing apparatus according to claim 1 , wherein the deterioration degree obtaining unit is further configured to obtain the deterioration degree from a deterioration degree holding unit configured to hold in advance information that indicates an image deterioration degree for each position of a captured image that is captured by an image capturing apparatus.

3 . The image processing apparatus according to claim 2 , wherein, the deterioration degree holding unit is further configured to hold the information that indicates the deterioration degree in association with a position and orientation between the target object and the image capturing apparatus.

4 . The image processing apparatus according to claim 2 , wherein the deterioration degree holding unit is further configured to hold information indicating a deterioration degree, of a captured image obtained by an image capturing apparatus by capturing the target object, at a position of each feature of an image of the target object in association with a feature that the three-dimensional model comprises that corresponds to the respective feature.

5 . The image processing apparatus according to claim 1 , wherein the deriving unit is further configured to correct the predetermined position and orientation to make a distance between the feature of the three-dimensional model and the feature of the target object that are associated by the associating unit small.

6 . The image processing apparatus according to claim 1 , wherein
the feature is an edge feature, the extraction unit is further configured to extract a plurality of edge features by performing edge detection processing on the captured image, and the associating unit is further configured to, for each of a plurality of edges that the three-dimensional model has, calculate an image position obtained by projecting the edge on a projection image based on the predetermined position and orientation, and associates an image position of edge feature of an image of the target object in the captured image with an image position of an edge of the three-dimensional model on the projection image which neighbors the edge feature of the image of the target object.

7 . The image processing apparatus according to claim 1 , wherein
the captured image is obtained by the image capturing apparatus capturing the target object, on which an illumination pattern is irradiated by an irradiation apparatus, the extraction unit is further configured to extract as the feature a three-dimensional position of a point on an image of the target object, based on a position of the irradiation apparatus, a position of the image capturing apparatus, and the illumination pattern, and the associating unit is further configured to associate the three-dimensional position of the point on the image of the target object and a three-dimensional position of a surface of the three-dimensional model that neighbors that three-dimensional position.

8 . The image processing apparatus according to claim 1 , wherein the deriving unit is further configured to derive the position and orientation of the target object with respect to the image capturing apparatus by correcting the predetermined position and orientation based on the result of the associating and the deterioration degree corresponding to a position at which the feature was extracted.

9 . The image processing apparatus according to claim 1 , further comprising a setting unit configured to set an extraction parameter used to extract the feature from the captured image in accordance with the deterioration degree,
wherein the extraction unit is further configured to extract the feature from the captured image by using the extraction parameter set by the setting unit.

10 . The image processing apparatus according to claim 9 , wherein the setting unit is further configured to set the extraction parameter so that a feature is extracted by filtering processing that uses a filter, wherein the higher the deterioration degree is, the larger the size of the filter is.

11 . The image processing apparatus according to claim 9 , wherein the setting unit is further configured to set the extraction parameter so that a feature is extracted by filtering processing after resizing the captured image, wherein the higher the deterioration degree is, the smaller the captured image is resized.

12 . The image processing apparatus according to claim 9 , wherein
the setting unit is further configured to set a plurality of extraction parameters in accordance with the deterioration degree, and the extraction unit is further configured to use each of the plurality of extraction parameters to extract a feature in accordance with filtering processing, and selects at least one extraction result from a plurality of extraction results in accordance with a response value of filtering processing.

13 . The image processing apparatus according to claim 1 , wherein the deterioration degree indicates at least one of a blur amount and a bokeh amount of the image.

14 . The image processing apparatus according to claim 1 , further comprising a deterioration degree calculation unit configured to calculate the deterioration degree by using the three-dimensional model of the target object, based on an image capturing condition of the target object according to the image capturing apparatus.

15 . The image processing apparatus according to claim 1 , further comprising a deterioration degree calculation unit configured to calculate the deterioration degree based on a captured image obtained by the image capturing apparatus capturing the target object.

16 . The image processing apparatus according to claim 1 , further comprising a deterioration degree calculation unit configured to estimate a captured image obtained by the image capturing apparatus capturing the target object using the three-dimensional model of the target object based on an image capturing condition of the target object according to the image capturing apparatus, and calculating the deterioration degree based on the estimated image.

17 . The image processing apparatus according to claim 1 , further comprising an image capturing unit configured to obtain the captured image by capturing the target object.

18 . The image processing apparatus according to claim 1 , further comprising:
an image capturing unit configured to obtain the captured image by capturing the target object; a robot arm comprising a movable shaft; and a control unit

Assignees

• Canon Kk

Inventors

• Miyatani Sonoko
• Suzuki Tatsuya
• Fujiki Masakazu
• Kobayashi Kazuhiko
• Suzuki Masahiro
• Hirota Yuichiro
• Kotake Daisuke

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/443

  by matching or filtering · CPC title

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• Y10S901/09

  Closed loop, sensor feedback controls arm movement · CPC title