Providing apparatus for augmented reality service, display apparatus and providing system for augmented reality service comprising the same

Patent metadata

Field	Value
Publication number	US-2017193700-A1
Application number	US-201615001414-A
Country	US
Kind code	A1
Filing date	Jan 20, 2016
Priority date	Jan 4, 2016
Publication date	Jul 6, 2017
Grant date	—

Abstract

A providing apparatus for an augmented reality service includes: a parameter calculating unit calculating camera parameters of a plurality of respective cameras; a mesh information processing unit converting point cloud information based on images obtained from the plurality of respective cameras into mesh information for the plurality of respective cameras and converting the mesh information into a world coordinate for a target space photographed by the plurality of cameras by using the camera parameters; a map generating unit generating a whole map for the target space by considering an area where the converted mesh information for the plurality of respective cameras is duplicated; and an augmentation processing unit augmenting a virtual object to the whole map.

First claim

What is claimed is:

1 . A providing apparatus for an augmented reality service, the providing apparatus, comprising:
a parameter calculating unit calculating camera parameters of a plurality of respective cameras; a mesh information processing unit converting point cloud information based on images obtained from the plurality of respective cameras into mesh information for the plurality of respective cameras and converting the mesh information into a world coordinate for a target space photographed by the plurality of cameras by using the camera parameters; a map generating unit generating a whole map for the target space by considering an area where the converted mesh information for the plurality of respective cameras is duplicated; and an augmentation processing unit augmenting a virtual object to the whole map.

2 . The providing apparatus of claim 1 , wherein the parameter calculating unit calculates the camera parameters by using the point cloud information based on the images obtained from the plurality of respective cameras.

3 . The providing apparatus of claim 1 , wherein the parameter calculating unit calculates internal parameters and external parameters of the plurality of respective cameras.

4 . The providing apparatus of claim 1 , wherein the map generating unit generates the whole map for the target space by simplifying the area where the converted mesh information for the plurality of respective cameras is duplicated.

5 . The providing apparatus of claim 1 , further comprising:
a communication unit transmitting information on the whole map, information on the virtual object, and processing information depending on an input of a user for the virtual object to another apparatus.

6 . The providing apparatus of claim 1 , wherein the plurality of cameras are RGB-D cameras.

7 . A display apparatus comprising:
a communication unit receiving world coordinate information of a target space and whole map information; a camera photographing the target space; a parameter calculating unit calculating camera parameters of the camera; a mesh information processing unit converting point cloud information based on an image obtained from the camera into mesh information and converting the mesh information into a world coordinate by using the camera parameters; a position estimating unit estimating the position of a photographing area of the camera on a whole map by using the converted mesh information and the whole map information; and an augmentation processing unit augmenting a virtual object to the photographing area.

8 . The display apparatus of claim 7 , wherein the communication unit further receives information on the virtual object.

9 . The display apparatus of claim 7 , wherein the augmentation processing unit augments a virtual object that matches the estimated photographing area by using the information on the virtual object.

10 . The display apparatus of claim 7 , wherein the parameter calculating unit calculates the camera parameters by using the point cloud information based on the image obtained from the camera.

11 . The display apparatus of claim 7 , wherein the parameter calculating unit calculates an internal parameter and an external parameter of the camera.

12 . The display apparatus of claim 7 , further comprising:
a display unit outputting the photographing area of the camera and the virtual object that matches the estimated photographing area.

13 . A providing system for an augmented reality service, the providing system comprising:
an augmented reality service providing apparatus converting point cloud information based on images obtained from the plurality of respective cameras into mesh information for the plurality of respective cameras, generating whole map information for a target space photographed by the plurality of cameras by using the mesh information for the plurality of cameras, and augmenting a virtual object on the whole map; and a display apparatus estimating a photographing area of a camera on the whole map based on the whole map information transferred from the augmented reality service providing apparatus and augmenting the virtual object to the estimated photographing area.

14 . The providing system of claim 13 , wherein the augmented reality service providing apparatus includes:
a parameter calculating unit calculating camera parameters of a plurality of respective cameras; a mesh information processing unit converting point cloud information based on images obtained from the plurality of respective cameras into mesh information for the plurality of respective cameras and converting the mesh information into a world coordinate for a target space photographed by the plurality of cameras by using the camera parameters; a map generating unit generating a whole map for the target space by considering an area where the converted mesh information for the plurality of respective cameras is duplicated; and an augmentation processing unit augmenting a virtual object to the whole map.

15 . The providing system of claim 13 , wherein the display apparatus includes:
a communication unit receiving world coordinate information of a target space and whole map information; a camera photographing the target space; a parameter calculating unit calculating camera parameters of the camera; a mesh information processing unit converting point cloud information based on an image obtained from the camera into mesh information and converting the mesh information into a world coordinate by using the camera parameter; a position estimating unit estimating the position of a photographing area of the camera on the whole map by using the converted mesh information and the whole map information; and an augmentation processing unit augmenting a virtual object to the photographing area.

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Jung Sung Uk
• Cho Hyun Woo

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title