Method and apparatus for determining camera location information and/or camera pose information according to a global coordinate system

Patent metadata

Field	Value
Publication number	US-9699375-B2
Application number	US-201314015610-A
Country	US
Kind code	B2
Filing date	Aug 30, 2013
Priority date	Apr 5, 2013
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

An approach is provided for processing and/or facilitating a processing of one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a global coordinate system. The approach involves causing, at least in part, an association of the camera location information, the camera pose information, or a combination thereof with the one or more images as meta-data information.

First claim

What is claimed is:

1. A method comprising facilitating a processing of, or processing, (1) data, (2) information, or (3) at least one signal, the (1) data, (2) information, (3) at least one signal based, at least in part, on the following:
a selection of one or more images from among one or more frames of at least one video sequence, wherein the one or more images represent, at least in part, a subset of the one or more frames;
a processing of the one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a local three-dimensional (3D) Cartesian coordinate system;
a processing of the one or more images to determine a local point cloud of one or more objects depicted in the one or more images, wherein the local point cloud is represented according to the local 3D Cartesian coordinate system;
at least one determination of one or more reference point clouds represented according to an Earth Centered, Earth Fixed (ECEF) 3D Cartesian coordinate system based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera;
a processing of at least one of the one or more reference point clouds represented according to the ECEF 3D Cartesian coordinate system to determine a correspondence of the local point cloud represented in the local 3D Cartesian coordinate system, wherein the correspondence maps the camera location information and the camera pose information from the local 3D Cartesian coordinate system to the ECEF 3D Cartesian coordinate system;
an association of the camera location information or the camera pose information represented according to the ECEF 3D Cartesian coordinate system with the one or more images as metadata information; and
post-processing of the at least one video sequence to extract 3D geo-augmented metadata from the one or more frames of the at least one video sequence based on the camera location or the camera pose information as represented according to the ECEF 3D Cartesian coordinate system and as associated in the metadata information,
wherein results of the post-processing of the at least one video sequence are displayed on a user display.

2. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:
a matching of the local three-dimensional point cloud to the one or more reference point clouds; and
a definition of the camera location information, the camera pose information, or a combination thereof according to the global coordinate system based, at least in part, on the matching.

3. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:
at least one determination of overlay information based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and
a rendering of the overlay information in the one or more images.

4. A method of claim 1 , wherein the (1) data, (2) information, (3) at least one signal are further based, at least in part, on the following:
at least one determination of popularity information associated with the camera location information, the camera pose information, or a combination thereof; and
a presentation of one or more representations of the popularity information in at least one user interface of the at least one camera.

5. A method of claim 4 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:
at least one determination of one or more representative images associated with the camera location, the camera pose information, or a combination thereof; and
a presentation of (a) the one or more representative images; (b) guidance information for navigating to one or more locations, for pointing the at least one camera to one or more targets, or a combination thereof associated with the one or more representative images; (c) one or more indicators for confirming that the at least one camera is at or near the one or more locations associated with the one or more representative images, pointing to the one or more targets, or a combination thereof; (d) or a combination thereof.

6. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:
at least one determination of one or more panoramic images based, at least in part, on the camera location information, the camera pose information, or a combination thereof;
at least one determination of field of view information for the one or more images based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and
an extension of the one or more images with at least a portion of the one or more panoramic images based, at least in part, on the field of view information.

7. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:
at least one determination of the one or more other images associated with other camera location information, other camera pose information, or a combination thereof based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and
an association of the one or more other images with the one or more images for interaction by at least one user during playback of the one or more images.

8. A method of claim 1 , wherein one or more other images are further determined based, at least in part, on one or more contextual criteria, and wherein the one or more contextual criteria include, at least in part, one or more temporal criteria.

9. An apparatus comprising:
at least one processor; and
at least one memory including computer program code for one or more programs,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,
cause, at least in part, a selection of one or more images from among one or more frames of the at least one video sequence, wherein the one or more images represent, at least in part, a subset of the one or more frames;
process and/or facilitate a processing of the one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a local three-dimensional (3D) Cartesian coordinate system and the one or more images are part of at least one video sequence;
process and/or facilitate a processing of the one or more images to determine a local point cloud of one or more objects depicted in the one or more images, wherein the local point cloud is represented according to the local 3D Cartesian coordinate system;
determine one or more reference point clouds represented according to an Earth Centered, Earth Fixed (ECEF) 3D Cartesian coordinate system based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera;
process and/or facilitate a processing of at least one of the one or more reference point clouds represented according to the ECEF 3D Cartesian coordinate system to determine a correspondence

Assignees

• Nokia Technologies Oy
• Nokia Technologies Oy

Inventors

• Fan Lixin
• Fu Junsheng
• Roimela Kimmo Tapio
• You Yu

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• H04N1/32128

  attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image (for information embedded in the image H04N1/32144) · CPC title

• H04N2201/3253

  Position information, e.g. geographical position at time of capture, GPS data · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title