Information processing apparatus, information processing method, and storage medium

Patent metadata

Field	Value
Publication number	US-11003939-B2
Application number	US-201916451361-A
Country	US
Kind code	B2
Filing date	Jun 25, 2019
Priority date	Jul 6, 2018
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

There is provided with an information processing apparatus. From an image capturing apparatus which can move, a captured image of an object is obtained. The position of the image capturing apparatus is derived using the captured image and a three-dimensional map. The three-dimensional map is corrected using information indicating a reliability of: information indicating the three-dimensional position of the feature included in a predefined area in the three-dimensional map held by the holding unit; and information indicating a three-dimensional position of another feature of the object, the information obtained based on the captured image from an expanded area of the predefined area in the three-dimensional map held by the holding unit.

First claim

What is claimed is:

1. An information processing apparatus comprising:
one or more processors programmed to cause the information processing apparatus to function as:
an obtaining unit configured to obtain, from an image capturing apparatus which can move, a captured image of an object;
a holding unit configured to hold a three-dimensional map containing information indicating a three-dimensional position of a feature of the object;
a deriving unit configured to derive the position of the image capturing apparatus using the captured image and the three-dimensional map; and
a correction unit configured to correct the three-dimensional map using information indicating a reliability of:
information indicating the three-dimensional position of the feature included in a predefined area in the three-dimensional map held by the holding unit; and
information indicating a three-dimensional position of another feature of the object, the information obtained based on the captured image from an expanded area of the predefined area in the three- dimensional map held by the holding unit.

2. The information processing apparatus according to claim 1 , wherein the one or more processors are further programmed to cause the information processing apparatus to function as a setting unit configured to set a reliability degree, as information indicating the reliability, to the information indicating the three-dimensional position of the feature.

3. The information processing apparatus according to claim 1 , wherein the correction unit is further configured to refrain from updating information indicating a three-dimensional position of a first feature, but to update information indicating a three-dimensional position of a second feature whose reliability is less than a reliability of the first feature.

4. The information processing apparatus according to claim 1 , wherein the correction unit is further configured to update the information indicating the three-dimensional position of the feature by an update amount corresponding to the reliability of the information indicating the three-dimensional position of the feature.

5. The information processing apparatus according to claim 1 , wherein
the information indicating the three-dimensional position of the feature is information indicating the three-dimensional position of the feature in a reference coordinate system defined in a space where the object is positioned, and
the correction unit is further configured to update the information indicating the three-dimensional position of the feature in the reference coordinate system.

6. The information processing apparatus according to claim 1 , wherein
the information indicating the three-dimensional position of the feature is a combination of position and orientation information of a viewpoint in a reference coordinate system defined in a space in which the object is positioned, and position information of the feature in a coordinate system that is defined relative to the viewpoint, and
the correction unit is further configured to update the position and orientation information of the viewpoint in the reference coordinate system.

7. The information processing apparatus according to claim 1 , wherein
the three-dimensional map includes information indicating an observation result of the object at each viewpoint out of a plurality of viewpoints, and
the correction unit is further configured to update the information indicating the three-dimensional position of the feature so that an error between the observation result observed at the viewpoint when the feature is at the three-dimensional position and an observation result actually observed at the viewpoint becomes smaller.

8. The information processing apparatus according to claim 2 , wherein the setting unit sets, to the information indicating the three-dimensional position of the feature included in the predefined area in the three-dimensional map, the reliability degree higher than that to the information indicating the three-dimensional position of the another feature of the object.

9. The information processing apparatus according to claim 2 , wherein
an assistance marker for measuring the position of the image capturing apparatus based on the captured image is arranged in a space where the object is positioned, and
the setting unit is further configured to set the reliability degree, based on information of the assistance marker, to the information indicating the three-dimensional position of the feature.

10. The information processing apparatus according to claim 7 , wherein the correction unit is further configured to update the information indicating the three-dimensional position of the feature so that an error between color information of the feature and the color information observed for the three-dimensional position at the viewpoint becomes smaller.

11. The information processing apparatus according to claim 7 , wherein the correction unit is further configured to update the information indicating the three-dimensional position of the feature so that an error between a direction from the viewpoint to the three-dimensional position and a direction from the viewpoint to the feature observed at the viewpoint becomes smaller.

12. The information processing apparatus according to claim 9 , wherein the setting unit is further configured to set the reliability degree, based on a number of the assistance marker in the captured image, to the information indicating the three-dimensional position of the feature generated based on the captured image.

13. An information processing apparatus comprising:
one or more processors programmed to cause the information processing apparatus to function as:
a holding unit configured to hold a three-dimensional map including, for each viewpoint out of a plurality of viewpoints, information indicating a three- dimensional position of an object with respect to the viewpoint, and position and orientation information of the viewpoint;
an obtaining unit configured to obtain, from an image capturing apparatus which can move, a captured image of the object from a new viewpoint;
a deriving unit configured to derive a position and orientation of the new viewpoint using the captured image and the three-dimensional map; and
a correction unit configured to correct the three-dimensional map using information indicating a reliability of:
position and orientation information of the new viewpoint, which is obtained based on the captured image from an expanded area of a predefined area in the three-dimensional map held by the holding unit; and
position and orientation information of a viewpoint included in the predefined area in the three-dimensional map.

14. An information processing apparatus comprising:
one or more processors programmed to cause the information processing apparatus to function as:
an obtaining unit configured to obtain, from an image capturing apparatus, a captured image of an object;
a holding unit configured to hold a pre-generated pre-map and a temporary map, wherein the pre-map includes information indicating a three-dimensional position of a feature of the object, and the temporary map is generated on the basis of the captured image;
a deriving unit configured to derive a position and orientation of the image capturing apparatus using the captured image, and the pre-map or the temporary map; and
a generation unit configured to generate, based on the captured image and the position and orientation of the image capturing apparatus, the temporary map which includes the information indicating the three-dimensional position of the feature of th

Assignees

• Canon Kk

Inventors

• Kotake Daisuke
• Katayama Akihiro
• Tomioka Makoto
• Kasuya Nozomu
• Yamada Takayuki
• Suzuki Masahiro
• Fujiki Masakazu

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/242

  by image rotation, e.g. by 90 degrees · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title

• G06K9/3208 Primary

  Physics · mapped topic