Determining Size Of Virtual Object

Patent metadata

Field	Value
Publication number	US-2019378289-A1
Application number	US-201716462450-A
Country	US
Kind code	A1
Filing date	Dec 1, 2017
Priority date	Dec 2, 2016
Publication date	Dec 12, 2019
Grant date	—

Abstract

A system and method are provided for determining a size of a virtual object in a virtual environment. The virtual object may represent an object in physical space. The system and method may obtain image data of the object from a camera, obtain camera metadata indicative of an angle of view of the camera, estimate an object distance between the object and the camera, and estimate a physical size of the object in physical space by determining an image size of the object in the image data, and determining a relation between the image size and the physical size of the object on the basis of the camera metadata and the object distance. Accordingly, the size of the virtual object in the virtual environment may be determined in accordance with the physical size of the object. Virtual objects may thus be given realistic relative sizes in the virtual environment. A further advantage may be that there is no need for a calibrated system of multiple cameras and a calibrated rendering environment.

First claim

1 . A method for determining a size of a virtual object in a virtual environment, the virtual object representing an object in physical space, comprising:
obtaining image data of the object from a camera; obtaining camera metadata indicative of an angle of view of the camera; estimating an object distance between the object and the camera; estimating a physical size of the object in physical space by:
i) determining an image size of the object in the image data, and
ii) determining a relation between the image size and the physical size of the object on the basis of the camera metadata and the object distance; and
determining the size of the virtual object for rendering in the virtual environment in accordance with the physical size of the object.

2 . The method according to claim 1 , wherein the virtual object in the virtual environment comprises a visual rendering of the image data of the object.

3 . The method according to claim 1 , wherein the obtaining the camera metadata comprises obtaining EXIF metadata, MPEG metadata or ONVIF metadata included in or associated with the image data.

4 . The method according to claim 1 , wherein the obtaining the camera metadata comprises:
accessing a database comprising camera metadata of different camera types; obtaining a type identifier of the camera; and retrieving at least part of the camera metadata on the basis of looking up the type identifier of the camera in the database.

5 . The method according to claim 1 , wherein the determining the object distance comprises obtaining auxiliary data which is indicative of the object distance, and wherein estimating the object distance between the object and the camera comprises analyzing the auxiliary data.

6 . The method according to claim 5 , wherein the auxiliary data comprises at least one of:
camera metadata indicative of a focal distance of the camera; depth data obtained from a depth sensor connected to the camera; further image data obtained from a further camera, wherein the image data and the further image data together represent stereoscopic image data of the object, and wherein estimating the object distance between the object and the camera comprises analyzing the stereoscopic image data; and directional audio data obtained from a microphone array connected to or comprised in the camera.

7 . The method according to claim 1 , wherein the virtual environment is a networked virtual environment, wherein the camera is connected to a system which is connected to a network, and wherein the method further comprises:
providing the image data from the system via the network to a network entity participating in the networked virtual environment, the network entity being a server hosting the virtual environment or a rendering device; and providing object size metadata from the system to the network entity to enable the network entity to establish the size of the virtual object in the virtual environment in accordance with a physical size of the object in physical space on the basis of the object size metadata.

8 . The method according to claim 7 , wherein the method is performed by the system, wherein the providing the object size metadata comprises:
providing, as the object size metadata, the size of the virtual object in the virtual environment as determined by the system.

9 . The method according to claim 7 , wherein the providing the object size metadata comprises providing the camera metadata and the object distance to the network entity to enable the network entity to:
estimate the physical size of the object in physical space on the basis of the camera metadata and the object distance; and determine the size of the virtual object in the virtual environment in accordance with the physical size of the object.

10 . A non-transitory computer-readable medium comprising object size metadata associated with image data of an object, wherein the image data is obtained from a camera, and wherein the object size metadata comprises:
camera metadata indicative of an angle of view of the camera; data indicative of an object distance between the object and the camera.

11 . A non-transitory computer-readable medium comprising object size metadata defining a size of a virtual object in a virtual environment, wherein the virtual object represents an object in physical space, and wherein the size of the virtual object in the virtual environment is in accordance with a physical size of the object as determined from image data of the object and camera metadata.

12 . A system configured to generate object size metadata associated with a virtual object in a virtual environment, the virtual object representing an object in physical space, the system comprising:
a camera interface to a camera for obtaining image data of the object; a memory comprising instruction data representing a set of instructions; a processor configured to communicate with the camera interface and the memory and to execute the set of instructions, wherein the set of instructions, when executed by the processor, cause the processor to:
obtain camera metadata indicative of an angle of view of the camera;
estimate an object distance between the object and the camera;
generate object size metadata on the basis of the camera metadata and the object distance to enable a rendering device or server to establish the size of the virtual object in the virtual environment in accordance with a physical size of the object in physical space on the basis of the object size metadata.

13 . The system according to claim 12 , wherein the set of instructions, when executed by the processor, causes the processor to:
estimate the physical size of the object in physical space by:
i) determining an image size of the object in the image data, and
ii) determining a relation between the image size and the physical size of the object on the basis of the camera metadata and the object distance;
determine the size of the virtual object in the virtual environment in accordance with the physical size of the object; and generate the object size metadata to comprise the size of the virtual object in the virtual environment as determined by the processor.

14 . The system according to claim 12 , wherein the virtual environment is a networked virtual environment, wherein the system further comprises a network interface to a network, wherein the processor is configured to communicate with the network interface, and wherein the set of instructions, when executed by the processor, cause the processor to:
provide the image data via the network interface to a network entity participating in the networked virtual environment, the network entity being a server hosting the virtual environment or a rendering device for rendering the virtual environment; and provide the object size metadata to the network entity to enable the network entity to establish the size of the virtual object in the virtual environment in accordance with the physical size of the object on the basis of the object size metadata.

15 . A network entity being a server for hosting a virtual environment or a rendering device for rendering the virtual environment, the virtual environment comprising a virtual object, the virtual object representing an object in physical space, the network entity comprising:
a network interface to a network for receiving:
image data of the object from a camera;
object size metadata indicative of a size of a virtual object in the virtual environment;
a memory comprising instruction data representing a set of instructions; a processor configured to communi

Assignees

• Koninklijke Kpn Nv
• Nederlanddse Organisatie Voor Toegepast Natuurwetenschappelijk Onderzoek Tno

Inventors

• Stokking Hans Maarten
• Prins Martin

Classifications

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/593

  from stereo images · CPC title

• G02B27/017

  Head mounted · CPC title

• G06T15/205

  Image-based rendering · CPC title