Authoring device and authoring method

Patent metadata

Field	Value
Publication number	US-2017316612-A1
Application number	US-201715492587-A
Country	US
Kind code	A1
Filing date	Apr 20, 2017
Priority date	Apr 28, 2016
Publication date	Nov 2, 2017
Grant date	—

Abstract

A method includes specifying a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point, correcting the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map, and controlling a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

First claim

What is claimed is:

1 . A method executed by a computer, the method comprising:
specifying a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point; correcting the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map; and controlling a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

2 . The method according to claim 1 , wherein the correcting includes:
calculating a first intersection of the capturing direction and an approximate plane based on a part of the plurality of map points defined in the second map, the part of the plurality of map points being located around a specific map point which is associated with the virtual object, and correcting the position of the virtual object to a position of the first intersection.

3 . The method according to claim 2 , wherein the correcting includes:
calculating a second intersection of the capturing direction and a plane that passes through a center of gravity of the approximate plane and is perpendicular to the approximate plane when an angle formed by a line normal to the approximate plane and the capturing direction is close to a right angle, and correcting the position of the virtual object to a position of the second intersection.

4 . The method according to claim 1 , wherein
the correcting includes determining whether the position of the virtual object is to be corrected based on an error between a position of a first map point of the first map and a second position of a second map point of the second map, and the first map point and the second map point are corresponding each other and located around the position of the virtual object.

5 . The method according to claim 1 , wherein the correcting includes determining whether the position of the virtual object is to be corrected, based on a density of map points of the first map, which is located around the position of the virtual object.

6 . The method according to claim 1 , further comprising:
determining a plurality of keyframes from among frames photographed by the camera, the plurality of keyframes including a first keyframe and a second keyframe that follows the first keyframe; and generating the second map, based on the first keyframe and the second keyframe.

7 . The method according to claim 1 , further comprising:
determining a plurality of keyframes from among frames photographed by the camera; and generating the second map, based on the plurality of keyframes, by bundle adjustment.

8 . The method according to claim 1 , further comprising:
determining a keyframe from among frames photographed by the camera; and generating the second map by finely adjusting a position of a map point of the first map based on the keyframe and a frame from among the frames which follows the keyframe.

9 . A non-transitory computer-readable storage medium storing a program that causes a computer to execute a process, the process comprising:
specifying a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point; correcting the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map; and controlling a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

10 . A device comprising:
a memory; and a processor coupled to the memory and configured to:
specify a position of a virtual object based on a position of a map point that is defined in a first map and indicates three-dimensional coordinates of a feature point,
correct the position of the virtual object based on positions of a plurality of map points defined in a second map and a capturing direction of a camera when the first map is changed to the second map that is different from the first map, and
control a display to display the virtual object, based on the corrected position of the virtual object and an image captured by the camera.

11 . The device according to claim 10 , wherein the processor is configured to:
calculate a first intersection of the capturing direction and an approximate plane based on a part of the plurality of map points defined in the second map, the part of the plurality of map points being located around a specific map point which is associated with the virtual object, and correct the position of the virtual object to a position of the first intersection.

12 . The device according to claim 11 , wherein the processor is configured to:
calculate a second intersection of the capturing direction and a plane that passes through a center of gravity of the approximate plane and is perpendicular to the approximate plane when an angle formed by a line normal to the approximate plane and the capturing direction is close to a right angle, and correct the position of the virtual object to a position of the second intersection.

13 . The device according to claim 10 , wherein
the processor is configured to determine whether the position of the virtual object is to be corrected based on an error between a position of a first map point of the first map and a second position of a second map point of the second map, and the first map point and the second map point are corresponding each other and located around the position of the virtual object.

14 . The device according to claim 10 , wherein the processor is configured to determine whether the position of the virtual object is to be corrected, based on a density of map points of the first map, which is located around the position of the virtual object.

15 . The device according to claim 10 , wherein the processor is configured to:
determine a plurality of keyframes from among frames photographed by the camera, the plurality of keyframes including a first keyframe and a second keyframe that follows the first keyframe, and generate the second map, based on the first keyframe and the second keyframe.

16 . The device according to claim 10 , wherein the processor is configured to:
determine a plurality of keyframes from among frames photographed by the camera, and generate the second map, based on the plurality of keyframes, by bundle adjustment.

17 . The device according to claim 10 , wherein the processor is configured to:
determine a keyframe from among frames photographed by the camera, and generate the second map by finely adjusting a position of a map point of the first map based on the keyframe and a frame from among the frames which follows the keyframe.

Assignees

• Fujitsu Ltd

Inventors

• Moteki Atsunori
• Yamaguchi Nobuyasu
• Yoshitake Toshiyuki

Classifications

• G06T2219/2016

  Rotation, translation, scaling · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T15/20

  Perspective computation · CPC title

• G06T7/73

  using feature-based methods · CPC title