Camera pose estimation device and control method

Patent metadata

Field	Value
Publication number	US-10088294-B2
Application number	US-201615098039-A
Country	US
Kind code	B2
Filing date	Apr 13, 2016
Priority date	Jun 11, 2015
Publication date	Oct 2, 2018
Grant date	Oct 2, 2018

Abstract

A method includes determining movement of a camera from a first time point when a first image has been captured to a second time point when a second image has been captured, performing first estimation processing for estimating a position and pose of the camera in the second time point based on image data at the time of capturing, a past image captured in the past, and a past position and pose of the camera at a time point when the past image has been captured, when the movement is not a translational movement and a rotation movement around an optical direction, and performing a second estimation processing for estimating the position and pose based on a feature descriptor of a feature point extracted from the second image and a feature descriptor of a map point accumulated when the movement is the translational movement or the rotational movement.

First claim

What is claimed is:

1. A method comprising:
determining movement of a camera from a first time point when a first image has been captured to a second time point when a second image has been captured based on the first image and the second image captured by the camera;
performing first estimation processing for estimating a position and pose of the camera in the second time point based on image data at a time of capturing the image data, a past image captured in the past, and a past position and pose of the camera at a time point when the past image has been captured, when the movement is not a translational movement and a rotation movement around an optical direction; and
performing a second estimation processing for estimating the position and pose of the camera at the second time point based on a feature descriptor of a feature point extracted from the second image and a feature descriptor of a map point accumulated in the past, when the movement is the translational movement or the rotational movement around the optical direction.

2. The method according to claim 1 , further comprising:
performing a third estimation processing for estimating the position and pose of the camera using another method prior to the first estimation processing or the second estimation processing; and
performing the first estimation processing or the second estimation processing when quality determined by the third estimation processing is lower than a threshold.

3. The method according to claim 2 , further comprising:
performing a fourth estimation processing for estimating the position and pose of the camera based on a marker when the quality determined by the third estimation processing is lower than a threshold and the marker, which has a predetermined shape, has been detected from the second image.

4. The method according to claim 1 , wherein the first estimation processing is processing that uses an image-to-image method.

5. The method according to claim 1 , wherein the second estimation processing is processing that uses an image-to-map method.

6. The method according to claim 1 , further comprising:
displaying additional information on the second image to be displayed on a display based on a result from the first estimation processing or the second estimation processing.

7. The method according to claim 6 , wherein the additional information is an augmented reality content.

8. A device comprising:
a memory; and
a processor coupled to the memory and configured to:
determine movement of a camera from a first time point when a first image has been captured to a second time point when a second image has been captured based on the first image and the second image captured by the camera,
perform first estimation processing for estimating a position and pose of the camera in the second time point based on image data at a time of capturing the image data, a past image captured in the past, and a past position and pose of the camera at a time point when the past image has been captured, when the movement is not a translational movement and a rotation movement around an optical direction, and
perform a second estimation processing for estimating the position and pose of the camera at the second time point based on a feature descriptor of a feature point extracted from the second image and a feature descriptor of a map point accumulated in the past, when the movement is the translational movement or the rotational movement around the optical direction.

9. The device according to claim 8 , wherein the processor is configured to:
perform a third estimation processing for estimating the position and pose of the camera using another method prior to the first estimation processing or the second estimation processing, and
perform the first estimation processing or the second estimation processing when quality determined by the third estimation processing is lower than a threshold.

10. The device according to claim 9 , wherein the processor is configured to perform a fourth estimation processing for estimating the position and pose of the camera based on a marker when the quality determined by the third estimation processing is lower than a threshold and the marker, which has a predetermined shape, has been detected from the second image.

11. The device according to claim 8 , wherein the first estimation processing is processing that uses an image-to-image method.

12. The device according to claim 8 , wherein the second estimation processing is processing that uses an image-to-map method.

13. The device according to claim 8 , wherein the processor is configured to display additional information on the second image to be displayed on a display based on a result from the first estimation processing or the second estimation processing.

14. The device according to claim 13 , wherein the additional information is an augmented reality content.

15. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a process, the process comprising:
determining movement of a camera from a first time point when a first image has been captured to a second time point when a second image has been captured based on the first image and the second image captured by the camera;
performing first estimation processing for estimating a position and pose of the camera in the second time point based on image data at a time of capturing the image data, a past image captured in the past, and a past position and pose of the camera at a time point when the past image has been captured, when the movement is not a translational movement and a rotation movement around an optical direction; and
performing a second estimation processing for estimating the position and pose of the camera at the second time point based on a feature descriptor of a feature point extracted from the second image and a feature descriptor of a map point accumulated in the past, when the movement is the translational movement or the rotational movement around the optical direction.

16. The non-transitory computer-readable storage medium according to claim 15 , the process further comprising:
performing a third estimation processing for estimating the position and pose of the camera using another method prior to the first estimation processing or the second estimation processing; and
performing the first estimation processing or the second estimation processing when quality determined by the third estimation processing is lower than a threshold.

17. The non-transitory computer-readable storage medium according to claim 16 , the process further comprising:
performing a fourth estimation processing for estimating the position and pose of the camera based on a marker when the quality determined by the third estimation processing is lower than a threshold and the marker, which has a predetermined shape, has been detected from the second image.

18. The non-transitory computer-readable storage medium according to claim 15 , wherein the first estimation processing is processing that uses an image-to-image method.

19. The non-transitory computer-readable storage medium according to claim 15 , wherein the second estimation processing is processing that uses an image-to-map method.

20. The non-transitory computer-readable storage medium according to claim 15 , the process further comprising:
displaying additional information on the second image to be displayed on a display based on a result from the first estimation processing or the second estimation proc

Assignees

• Fujitsu Ltd

Inventors

• Moteki Atsunori
• Yamaguchi Nobuyasu
• Yoshitake Toshiyuki

Classifications

• G01B11/002 Primary

  for measuring two or more coordinates · CPC title

• G06T2207/30204

  Marker · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T2207/30244

  Camera pose · CPC title