Apparatus and method for estimating orientation of camera

Patent metadata

Field	Value
Publication number	US-9542745-B2
Application number	US-201514611330-A
Country	US
Kind code	B2
Filing date	Feb 2, 2015
Priority date	Feb 25, 2014
Publication date	Jan 10, 2017
Grant date	Jan 10, 2017

Abstract

An orientation estimation apparatus estimates a position and an orientation of an image capturing unit based on positions of a marker on the image and in the real space when the maker is detected; and based on positions of feature points on the captured image and in the real space, and initial values of the position and the orientation of the unit when the maker is not detected. Further, the apparatus calculates a failure index from a first image acquired while the position and the orientation are estimated by the position of marker, where the failure index indicates a possibility that the detection of the marker is failed. When the failure index satisfies a shift-preparation-start reference, the initial values are set from the position and the orientation estimated by the position of the marker on the image having been acquired in a predetermined period before the first image.

First claim

What is claimed is:

1. An orientation estimation apparatus, comprising:
at least one processor; and
at least one memory which stores a plurality of instructions, which when executed by the at least one processor, cause the at least one processor to execute:
creating, by an image capturing unit, an image of a photographed range at a predetermined photographing cycle;
storing therein a position of a marker installed in advance in a real space and positions of a plurality of feature points in a surrounding of the marker in the real space;
detecting the marker from the image;
estimating, based on a position of the marker on the image and the position of the marker in the real space, a position and an orientation of the image capturing unit;
detecting the plurality of feature points from the image;
estimating the position and the orientation of the image capturing unit based on positions of the plurality of feature points on the image, the positions of the plurality of feature points in the real space, and initial values of the position and the orientation of the image capturing unit;
detecting, when the marker is not detected from the image, the plurality of feature points with respect to the image and a subsequent image to be acquired after the image, and estimating the position and the orientation of the image capturing unit;
calculating, from a first image acquired while the position and the orientation of the image capturing unit are estimated based on the position of the marker on the image and the position of the marker in the real space, at least one failure index indicating a possibility that the detection of the marker is failed, and determining whether or not the at least one failure index satisfies a shift preparation start reference; and
setting, when the at least one failure index satisfies the shift preparation start reference, the initial values of the position and the orientation of the image capturing unit from the position and the orientation of the image capturing unit estimated based on the position of the marker on the image having been acquired in a predetermined period before the first image,
wherein the marker is determined to not be detected from the image based on the calculated at least one failure index.

2. The orientation estimation apparatus according to claim 1 , wherein the calculating calculates, as the failure index, at least one out of a position of the marker on the first image or a traveling speed of the marker, an area of the marker on the image, an angle made between a normal line direction of the marker and an optical axis direction of the image capturing unit, and reliability of the detection of the marker.

3. The orientation estimation apparatus according to claim 1 , wherein the setting sets, when the at least one failure index satisfies the shift preparation start reference, a position and an orientation of the image capturing unit that are estimated by applying a prediction filter to the position and the orientation of the image capturing unit that are estimated based on the position of the marker on the image acquired in the predetermined period as the initial values.

4. The orientation estimation apparatus according to claim 1 , wherein the at least one processor further executes:
calculating, from a second image acquired while the position and the orientation of the image capturing unit are estimated based on the positions of the plurality of feature points on the image, the positions of the plurality of feature points in the real space, and the initial values of the position and the orientation of the image capturing unit, at least one success index indicating a possibility that the detection of the marker is succeeded, determines whether or not the at least one success index satisfies a re-shift preparation start reference, and detecting the marker from the second image; and
detecting, when the marker is detected from the second image, the marker with respect to a subsequent image to be acquired after the second image, and estimating the position and the orientation of the image capturing unit.

5. The orientation estimation apparatus according to claim 4 , wherein the at least one processor further executes:
setting a search range that includes the marker on the second image obtained by projecting the position of the marker in the real space onto the second image based on the position and the orientation of the image capturing unit that are estimated and is smaller than the second image, and detecting the marker within the search range.

6. The orientation estimation apparatus according to claim 4 , wherein the at least one processor further executes:
calculating, as the success index, at least one out of a position of the marker on the second image obtained by projecting the position of the marker in the real space onto the second image based on the position and the orientation of the image capturing unit that are estimated, an area of the marker on the second image, and the angle made between the normal line direction of the marker and the optical axis direction of the image capturing unit.

7. An orientation estimation method, comprising:
detecting a marker from an image acquired from an image capturing unit that creates the image of a photographed range at a predetermined photographing cycle;
estimating, based on a position of the marker on the image and a position of the marker in a real space, a position and an orientation of the image capturing unit;
detecting, when the marker is not detected from the image, a plurality of feature points from the image and a subsequent image to be acquired after the image;
estimating the position and the orientation of the image capturing unit based on positions of the plurality of feature points on the image, positions of the plurality of feature points in the real space, and initial values of the position and the orientation of the image capturing unit;
calculating, from a first image acquired while the position and the orientation of the image capturing unit are estimated based on the position of the marker on the image and the position of the marker in the real space, at least one failure index indicating a possibility that the detection of the marker is failed, and determining whether or not the at least one failure index satisfies a shift preparation start reference; and
setting, when the at least one failure index satisfies the shift preparation start reference, the initial values of the position and the orientation of the image capturing unit from the position and the orientation of the image capturing unit estimated based on the position of the marker on the image having been acquired in a predetermined period before the first image,
wherein the marker is determined to not be detected from the image based on the calculated at least one failure index.

8. A non-transitory computer-readable recording medium storing an orientation estimation computer program for causing a computer to execute a process, the process comprising:
detecting a marker from an image acquired from an image capturing unit that creates the image of a photographed range at a predetermined photographing cycle;
estimating, based on a position of the marker on the image and a position of the marker in a real space, a position and an orientation of the image capturing unit;
detecting, when the marker is not detected from the image, a plurality of feature points from the image and a subsequent image to be acquired after the image;
estimating the position and the orientation of the image capturing unit based on positions of the plurality of feature points on the image, positions of the plurality of feature points in the real space, and initial values of the position and the orient

Assignees

• Fujitsu Ltd

Inventors

• Moteki Atsunori
• Yamaguchi Nobuyasu
• Matsuda Takahiro

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30204

  Marker · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title

• G06K9/3216

  Physics · mapped topic