Method, system, and computer program for recognizing position and attitude of object imaged by camera

Patent metadata

Field	Value
Publication number	US-12586242-B2
Application number	US-202318171786-A
Country	US
Kind code	B2
Filing date	Feb 21, 2023
Priority date	Feb 22, 2022
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

A method of the present disclosure includes (a) extracting distinctive features used for respectively distinguishing a plurality of similar attitudes from which images similar to one another are obtained using a simulation model of an object, (b) capturing an object image of the object using a camera, (c) estimating a position and an attitude of the object using the object image, and (d) when the estimated attitude corresponds to one of the plurality of similar attitudes, determining the one of the plurality of similar attitudes as the attitude of the object using the distinctive features.

First claim

What is claimed is:

1 . A method of recognizing a position and an attitude of an object imaged by a camera, comprising:
(a) extracting distinctive features used for respectively distinguishing a plurality of similar attitudes from which images similar to one another are obtained using a simulation model of the object, the extracting of the distinctive features including:
placing the simulation model at a center of a virtual sphere;
setting a plurality of viewpoints for viewing the simulation model at equal intervals along 360 degrees on an outer surface of the virtual sphere;
calculating a similarity of every two attitudes of a plurality of attitudes of the simulation model between viewing from one viewpoint of the plurality of viewpoints and viewing from another viewpoint of the plurality of viewpoints for every viewpoint of the plurality of viewpoints;
extracting the plurality of similar attitudes, each of the plurality of similar attitudes having the similarity that is equal to or more than a threshold value; and
extracting the distinctive features based on shapes and rotations of the plurality of similar attitudes;
(b) capturing an object image of the object using the camera; (c) estimating the position and the attitude of the object using the object image; and (d) when the estimated attitude corresponds to first attitude of the plurality of similar attitudes, determining the first attitude of the plurality of similar attitudes as the attitude of the object using the distinctive features.

2 . The method according to claim 1 , wherein
the distinctive features are shape distinctive features on shapes of characteristic portions of the object imaged by the camera in the respective plurality of similar attitudes, and the step (d) includes extracting object feature information corresponding to the shape distinctive features from the object image, and determining the first attitude having the shape distinctive feature of the plurality of similar attitudes closest to the object feature information as the attitude of the object.

3 . The method according to claim 1 , wherein
the plurality of similar attitudes are attitudes formed by rotation of the object around one rotation axis, the distinctive features include a template showing the object projected on a projection surface orthogonal to the rotation axis, and the step (d) includes (d1) obtaining a rotation angle between the template and the object in the object image, and (d2) selecting one of the plurality of similar attitudes using the rotation angle.

4 . The method according to claim 3 , wherein
the step (d2) includes respectively performing Fourier transformation and transformation into polar coordinates on the object image and the template and obtaining a first feature spectrum of the object image and a second feature spectrum of the template at the polar coordinates, and determining the rotation angle using an angular difference in polar coordinates between a peak position of the first feature spectrum and a peak position of the second feature spectrum.

5 . The method according to claim 1 , further comprising:
(e) causing a robot to perform an operation with respect to the object based on the estimated position of the object and one of the estimated attitude of the object and the first attitude of the plurality of similar attitudes.

6 . A system of recognizing a position and an attitude of an object, comprising:
a camera configured to capture an object image of the object; a memory configured to store a program; and a processor configured to execute the program so as to:
extract distinctive features used for respectively distinguishing a plurality of similar attitudes from which images similar to one another are obtained using a simulation model of the object, the extracting of the distinctive features including:
placing the simulation model at a center of a virtual sphere;
setting a plurality of viewpoints for viewing the simulation model at equal intervals along 360 degrees on an outer surface of the virtual sphere;
calculating a similarity of every two attitudes of a plurality of attitudes of the simulation model between viewing from one viewpoint of the plurality of viewpoints and viewing from another viewpoint of the plurality of viewpoints for every viewpoint of the plurality of viewpoints;
extracting the plurality of similar attitudes, each of the plurality of similar attitudes having the similarity that is equal to or more than a threshold value; and
extracting the distinctive features based on shapes and rotations of the plurality of similar attitudes;
(b) causing the camera to capture the object image of the object; (c) estimating the position and the attitude of the object using the object image; and (d) when the estimated attitude corresponds to first attitude of the plurality of similar attitudes, processing of determining the first attitude of the plurality of similar attitudes as the attitude of the object using the distinctive features.

7 . The system according to claim 6 ,
wherein the processor is further configured to perform:
(e) causing a robot to perform an operation with respect to the object based on the estimated position of the object and one of the estimated attitude of the object and the first attitude of the plurality of similar attitudes.

8 . A computer program product embodying computer readable instructions stored on a non-transitory computer-readable medium for controlling a processor to execute processing of recognizing a position and an attitude of an object imaged by a camera, the processor is configured to perform the steps of:
(a) processing of extracting distinctive features used for respectively distinguishing a plurality of similar attitudes from which images similar to one another are obtained using a simulation model of the object, the extracting of the distinctive features including: placing the simulation model at a center of a virtual sphere; setting a plurality of viewpoints for viewing the simulation model at equal intervals along 360 degrees on an outer surface of the virtual sphere; calculating a similarity of every two attitudes of a plurality of attitudes of the simulation model between viewing from one viewpoint of the plurality of viewpoints and viewing from another viewpoint of the plurality of viewpoints for every viewpoint of the plurality of viewpoints; extracting the plurality of similar attitudes, each of the plurality of similar attitudes having the similarity that is equal to or more than a threshold value; and extracting the distinctive features based on shapes and rotations of the plurality of similar attitudes; (b) processing of causing the camera to capture an object image of the object; (c) processing of estimating the position and the attitude of the object using the object image; and (d) when the estimated attitude corresponds to first attitude of the plurality of similar attitudes, processing of determining the first attitude of the plurality of similar attitudes as the attitude of the object using the distinctive features.

9 . The computer program product according to claim 8 ,
wherein the processor is further configured to perform:
(e) processing of causing a robot to perform an operation with respect to the object based on the estimated position of the object and one of the estimated attitude of the object and the first attitude of the plurality of similar attitudes.

Assignees

• Seiko Epson Corp

Inventors

• Hayashi Masaki
• Yao Jian
• Mukherjee Dibyendu
• Rotenstein Andrei Mark

Classifications

• B25J9/1671

  characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• B25J19/023

  including video camera means · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06T2207/30164

  Workpiece; Machine component · CPC title