Information processing apparatus, information processing method, and storage medium

Patent metadata

Field	Value
Publication number	US-11830216-B2
Application number	US-202117228824-A
Country	US
Kind code	B2
Filing date	Apr 13, 2021
Priority date	Jul 6, 2018
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

There is provided with an information processing apparatus. From an image capturing apparatus which can move, a captured image of an object is obtained. The position of the image capturing apparatus is derived using the captured image and a three-dimensional map. The three-dimensional map is corrected using information indicating a reliability of: information indicating the three-dimensional position of the feature included in a predefined area in the three-dimensional map held by the holding unit; and information indicating a three-dimensional position of another feature of the object, the information obtained based on the captured image from an expanded area of the predefined area in the three-dimensional map held by the holding unit.

First claim

What is claimed is:

1. An information processing apparatus comprising one or more processors and one or more memories storing a program which causes the one or more processors to function as:
an obtaining unit configured to obtain an image, wherein an image capturing apparatus captures the image while moving;
an acquisition unit configured to acquire a three-dimensional map, the three-dimensional map having first element information including the image and a first position and orientation of the image capturing apparatus when the image capturing apparatus captured the image; and
a correction unit configured to correct a second position and orientation of the image capturing apparatus when the image capturing apparatus captured a new image included in a second element information while fixing and not correcting the first position and orientation included in the first element information in a case where a part of feature points derived from the image included in the first element information is common to a part of feature points derived from the new image, wherein the second element information is different from a plurality of pieces of the first element information and the second element information includes the new image and the second position and orientation of the image capturing apparatus.

2. The information processing apparatus according to claim 1 , wherein the program further causes the one or more processors to function as a setting unit configured to set a reliability degree to the first position and orientation and the second position and orientation.

3. The information processing apparatus according to claim 2 , wherein the setting unit is further configured to set, to the first position and orientation, the reliability degree higher than that to the second position and orientation.

4. The information processing apparatus according to claim 1 , wherein the correction unit is further configured to update the three-dimensional map by adding the corrected second position and orientation and the new image.

5. The information processing apparatus according to claim 1 , wherein the updating unit is further configured to update the second position and orientation in a reference coordinate system defined in a space where an object is positioned.

6. The information processing apparatus according to claim 1 , wherein the reliability indicates a reliability of a position and orientation in relation to a true position and orientation of the image capturing apparatus in a reference coordinate system defined in a space where an object is positioned.

7. The information processing apparatus according to claim 1 , wherein the second position and orientation is estimated based on the new image obtained by capturing an object outside a predefined area in the three-dimensional map.

8. The information processing apparatus according to claim 1 , wherein the image capturing apparatus is a stereo camera.

9. The information processing apparatus according to claim 8 , wherein the updating unit is further configured to fix the first position and orientation and update the second position and orientation such that a reprojection error of the three-dimensional position information of a feature point commonly included in the image captured at the first position and information and the image captured at the second position and orientation is reduced.

10. An information processing apparatus comprising one or more processors and one or more memories storing a program which causes the one or more processors to function as:
an obtaining unit configured to obtain an image, wherein an image capturing apparatus captures the image while moving;
an acquisition unit configured to acquire a three-dimensional map, the three-dimensional map having first element information including the image and a first position and orientation of the image capturing apparatus when the image capturing apparatus captured the image; and
a correction unit configured to correct a second position and orientation of the image capturing apparatus when the image capturing apparatus captured a new image included in a second element information while fixing and not correcting the first position and orientation included in the first element information, such that a reprojection error of three-dimensional position information of a feature point commonly included in the image and the new image is reduced, wherein the second element information is different from a plurality of pieces of the first element information and the second element information includes the new image and the second position and orientation of the image capturing apparatus.

11. The information processing apparatus according to claim 10 , wherein the correction unit is further configured to update the three-dimensional map by adding the new image and the corrected second position and orientation.

12. An information processing apparatus comprising one or more processors and one or more memories storing a program which causes the one or more processors to function as:
an obtaining unit configured to obtain sensing information obtained from sensing of an object by a sensor apparatus while the sensor apparatus moves; and
an acquisition unit configured to acquire a three-dimensional map, the three-dimensional map having first element information including the sensing information and a first position and orientation of the sensor apparatus when the sensor apparatus sensed the object to obtain the sensing information; and
a correction unit configured to correct a second position and orientation of the sensor apparatus when the sensor apparatus sensed the object to obtain new sensing information included in a second element information while fixing and not correcting the first position and orientation included in the first element information in a case where a part of feature points derived from the sensing information included in the first element information is common to a part of feature points derived from the new sensing information, wherein the second element information is different from a plurality of pieces of the first element information and the second element information includes the new sensing information and the second position and orientation of the sensor apparatus.

13. The information processing apparatus according to claim 12 , wherein the sensor apparatus is a Lidar (Laser imaging detection and ranging) and the sensing information is distance information.

14. A method comprising:
obtaining an image, wherein an image capturing apparatus captures the image while moving;
acquiring a three-dimensional map, the three-dimensional map having first element information including the image and a first position and orientation of the image capturing apparatus when the image capturing apparatus captured the image; and
correcting a second position and orientation of the image capturing apparatus when the image capturing apparatus captured a new image included in a second element information while fixing and not correcting the first position and orientation included in the first element information in a case where a part of feature points derived from the image included in the first element information is common to a part of feature points derived from the new image, wherein the second element information is different from a plurality of pieces of the first element information and the second element information includes the new image and the second position and orientation of the image capturing apparatus.

15. A method comprising:
obtaining an image, wherein an image capturing apparatus captures the image while mov

Assignees

• Canon Kk

Inventors

• Kotake Daisuke
• Katayama Akihiro
• Tomioka Makoto
• Kasuya Nozomu
• Yamada Takayuki
• Suzuki Masahiro
• Fujiki Masakazu

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06V10/242

  by image rotation, e.g. by 90 degrees · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title