Information processing apparatus, information processing method, robot system, and non-transitory computer-readable storage medium

Patent metadata

Field	Value
Publication number	US-11094082-B2
Application number	US-201916535469-A
Country	US
Kind code	B2
Filing date	Aug 8, 2019
Priority date	Aug 10, 2018
Publication date	Aug 17, 2021
Grant date	Aug 17, 2021

Abstract

A plurality of verification position/orientation candidates for a target object is set. A common structure model including a geometric feature of a part, among geometric features of a reference model representing a three-dimensional shape of the target object, that is common among the candidates is generated. An image including the target object is obtained. A position/orientation of the target object is estimated by verifying the common structure model and the reference model arranged at the plurality of verification position/orientation candidates, against the image.

First claim

What is claimed is:

1. An information processing apparatus comprising:
one or more processors programmed to cause the information processing apparatus to function as:
a setting unit configured to set matching positions/orientations of respective models with a shape similar to that of a target object;
a generating unit configured to generate a common structure model including geometric features of parts, among geometric features of a reference model representing a three-dimensional shape of the target object, that is common among the models that the setting unit sets the matching positions/orientations;
an obtaining unit configured to obtain an image including the target object; and
an estimating unit configured to estimate a position/orientation of the target object by matching the image with the reference model which is arranged by using the matching positions/orientations and a position/orientation of a part of the target object estimated based on the image and the common structure model.

2. The information processing apparatus according to claim 1 ,
wherein the estimating unit calculates an approximate position/orientation of the target object by aligning an image feature detected from the image with the common structure model, and judges the position/orientation of the target object on the basis of the approximate position/orientation, the candidates, and the reference model, or judges the position/orientation of the target object after aligning the reference model with the image feature.

3. The information processing apparatus according to claim 1 ,
wherein the generating unit aligns the reference model arranged at a first position/orientation with the reference model arranged at a second position/orientation generated on the basis of the candidates, and extracts, from a geometric feature group of the reference model arranged at the first position/orientation, a geometric feature group of a part near which a geometric feature group of the reference model arranged at the second position/orientation is present.

4. The information processing apparatus according to claim 1 ,
wherein the generating unit aligns each of the reference models arranged at two or more positions/orientations generated on the basis of the candidates with an image feature detected from the image, and extracts, from a geometric feature group associated with the image feature, a geometric feature group of a part near which a geometric feature group of the reference models arranged at one or more different positions/orientations is present.

5. The information processing apparatus according to claim 1 ,
wherein of the geometric features of the reference model, the generating unit extracts a geometric feature aside from a part representing a fine structure of a shape.

6. The information processing apparatus according to claim 1 , wherein the one or more processors are further programmed to cause the information processing apparatus to function as:
a display control unit configured to cause the reference model and the common structure model to be displayed.

7. The information processing apparatus according to claim 6 , wherein the one or more processors are further programmed to cause the information processing apparatus to function as:
an accepting unit configured to accept a user operation,
wherein the display control unit arranges and displays the reference model on the basis of the candidates in response to the user operation.

8. The information processing apparatus according to claim 7 ,
wherein the accepting unit accepts a threshold designating a nearby range used when determining whether or not, among a geometric feature group of the reference model arranged at the first position/orientation, a geometric feature group of the reference model arranged at the second position/orientation generated on the basis of the candidates is present in the vicinity, and a threshold for determining whether or not a geometric feature of the reference model indicates a part representing a fine structure.

9. The information processing apparatus according to claim 3 , wherein the one or more processors are further programmed to cause the information processing apparatus to function as:
a first determination unit configured to determine, on the basis of a resolution of the image, a threshold designating a nearby range used when determining whether or not, among a geometric feature group of the reference model arranged at the first position/orientation, a geometric feature group of the reference model arranged at the second position/orientation generated on the basis of the candidates is present in the vicinity, and a threshold for determining whether or not a geometric feature of the reference model indicates a part representing a fine structure.

10. The information processing apparatus according to claim 9 , wherein the one or more processors are further programmed to cause the information processing apparatus to function as:
a second determination unit configured to determine, on the basis of a resolution of the image, whether or not the estimating unit verifies using the common structure model.

11. An information processing method carried out by an information processing apparatus, the method comprising:
setting matching positions/orientations of respective models with a shape similar to that of a target object;
generating a common structure model including geometric features of parts, among geometric features of a reference model representing a three-dimensional shape of the target object, that is common among the models that the setting unit sets the matching positions/orientations;
obtaining an image including the target object; and
estimating a position/orientation of the target object by matching the image with the reference model which is arranged by using at the matching positions/orientations and a position/orientation of a part of the target object estimated based on the image and the common structure model.

12. A robot system comprising:
an image capturing apparatus that captures an image;
an information processing apparatus including:
one or more processors programmed to cause the information processing apparatus to function as:
a setting unit configured to set matching positions/orientations of respective models with a shape similar to that of a target object;
a generating unit configured to generate a common structure model including geometric features of parts, among geometric features of a reference model representing a three-dimensional shape of the target object, that is common among the models that the setting unit sets the matching positions/orientations;
an obtaining unit configured to obtain an image including the target object; and
an estimating unit configured to estimate a position/orientation of the target object by matching the image with the reference model which is arranged by using the matching positions/orientations and a position/orientation of a part of the target object estimated based on the image and the common structure model; and

a robot that grips the target object on the basis of an estimation result of the position/orientation of the target object obtained by the information processing apparatus.

13. A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as:
a setting unit configured to set positions/orientations of respective models with a shape similar to that of a target object;
a generating unit configured to generate a common structure model including geometric feature of parts, among geometric features of a reference model representing a three-dimensional shape

Assignees

• Canon Kk

Inventors

• Yamazaki Fukashi
• Kotake Daisuke

Classifications

• G06V20/64 Primary

  Three-dimensional [3D] objects · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06T7/75 Primary

  involving models · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06K9/00664

  Physics · mapped topic