Image processing device, image processing method, and non-transitory computer-readable storage medium that determines camera position based on a comparison of estimated postures

Patent metadata

Field	Value
Publication number	US-10102647-B2
Application number	US-201615131326-A
Country	US
Kind code	B2
Filing date	Apr 18, 2016
Priority date	Jun 16, 2015
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

A device includes a processor configured to execute a first estimation process including detecting a marker from a captured image, and estimating first position and posture of a camera at a time when the captured image is captured, based on a shape of the marker, execute a second estimation process including obtaining a map point in a three-dimensional space from the memory, and estimating second position and posture of the camera at the time when the captured image is captured, based on a correspondence between a projection point in which the map point is projected on the captured image and a characteristic point extracted from the captured image, and select the first position and posture or the second position and posture based on a result of comparison between a first translational component of the first position and posture and a second translational component of the second position and posture.

First claim

What is claimed is:

1. A computer comprising:
a memory; and
a central processing unit coupled to the memory and configured to:
execute a first estimation process including detecting a marker having a predetermined shape from a captured image obtained from a camera, and estimating first position and posture of the camera at a time when the captured image is captured, based on a shape of the marker,
execute a second estimation process including obtaining a map point in a three-dimensional space from the memory, and estimating second position and posture of the camera at the time when the captured image is captured, based on a correspondence between a projection point in which the map point is projected on the captured image and a characteristic point extracted from the captured image, and
select the first position and posture or the second position and posture based on a result of comparison between a first translational component of the first position and posture and a second translational component of the second position and posture,
wherein the first position and posture are selected when a difference between the first translational component and the second translational component is a predetermined value or greater, and
the second position and posture are selected when the difference between the first translational component and the second translational component is less than the predetermined value.

2. The computer according to claim 1 , wherein object information is displayed on the captured image according to the selected first position and posture or second position and posture.

3. The computer according to claim 1 , wherein the central processing unit is configured to evaluate an accuracy of the second position and posture when the difference between the first translational component and the second translational component is less than the predetermined value.

4. The computer according to claim 3 , wherein the central processing unit is configured to
set the first position and posture as an initial value of the position and posture of the camera when the difference between the first translational component and the second translational component is the predetermined value or greater, or when the accuracy of the second position and posture is less than a threshold value, and
recalculate the second position and posture using the initial value.

5. An image processing method executed by a computer, the image processing method comprising:
executing a first estimation process including detecting a marker having a predetermined shape from a captured image obtained from a camera, and estimating first position and posture of the camera at a time when the captured image is captured, based on a shape of the marker;
executing a second estimation process including obtaining a map point in a three-dimensional space from a memory, and estimating second position and posture of the camera at the time when the captured image is captured, based on a correspondence between a projection point in which the map point is projected on the captured image and a characteristic point extracted from the captured image; and
selecting the first position and posture or the second position and posture based on a result of comparison between a first translational component of the first position and posture and a second translational component of the second position and posture,
wherein the selecting selects the first position and posture when a difference between the first translational component and the second translational component is a predetermined value or greater, and
the selecting selects the second position and posture when the difference between the first translational component and the second translational component is less than the predetermined value.

6. The image processing method according to claim 5 , wherein object information is displayed on the captured image according to the selected first position and posture or second position and posture.

7. The image processing method according to claim 5 , further comprising:
evaluating an accuracy of the second position and posture when the difference between the first translational component and the second translational component is less than the predetermined value.

8. The image processing method according to claim 7 , further comprising:
setting the first position and posture as an initial value of the position and posture of the camera when the difference between the first translational component and the second translational component is the predetermined value or greater, or when the accuracy of the second position and posture is less than a threshold value, and
the executing of the second estimation process recalculates the second position and posture using the initial value.

9. A non-transitory computer-readable storage medium storing an image processing program causing a computer to execute a process, the process comprising:
executing a first estimation process including detecting a marker having a predetermined shape from a captured image obtained from a camera, and estimating first position and posture of the camera at a time when the captured image is captured, based on a shape of the marker;
executing a second estimation process including obtaining a map point in a three-dimensional space from a memory, and estimating second position and posture of the camera at the time when the captured image is captured, based on a correspondence between a projection point in which the map point is projected on the captured image and a characteristic point extracted from the captured image; and
selecting the first position and posture or the second position and posture based on a result of comparison between a first translational component of the first position and posture and a second translational component of the second position and posture,
wherein the selecting selects the first position and posture when a difference between the first translational component and the second translational component is a predetermined value or greater, and
the selecting selects the second position and posture when the difference between the first translational component and the second translational component is less than the predetermined value.

10. The non-transitory computer-readable storage medium according to claim 9 , wherein object information is displayed on the captured image according to the selected first position and posture or second position and posture.

11. The non-transitory computer-readable storage medium according to claim 9 , wherein the process further comprising:
evaluating an accuracy of the second position and posture when the difference between the first translational component and the second translational component is less than the predetermined value.

12. The non-transitory computer-readable storage medium according to claim 11 , wherein the process further comprising:
setting the first position and posture as an initial value of the position and posture of the camera when the difference between the first translational component and the second translational component is the predetermined value or greater, or when the accuracy of the second position and posture is less than a threshold value, and
the executing of the second estimation process recalculates the second position and posture using the initial value.

Assignees

• Fujitsu Ltd

Inventors

• Yamaguchi Nobuyasu
• Moteki Atsunori

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/30204

  Marker · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title