Apparatus and method for generating virtual reality content based on non-virtual reality content

Patent metadata

Field	Value
Publication number	US-9811911-B2
Application number	US-201414585034-A
Country	US
Kind code	B2
Filing date	Dec 29, 2014
Priority date	Dec 29, 2014
Publication date	Nov 7, 2017
Grant date	Nov 7, 2017

Abstract

A method for generating virtual reality (VR) content includes obtaining source information from original media content. retrieving characteristic information stored in a first memory based on the obtained source information wherein the retrieved characteristic information related to the original media content, generating the VR content by combining the retrieved characteristics information and the obtained source information, wherein at least one feature of the retrieved characteristic information is mapped to a corresponding at least one feature of the obtained source information, and outputting the generated VR content and the original media content to be reproduced to a viewer.

First claim

What is claimed is:

1. A method for generating virtual reality (VR) content, the method comprising:
identifying an object in original media content;
generating metadata of the identified object;
searching for images of the identified object using the generated metadata of the identified object;
receiving one or more images depicting the identified object from a plurality of different viewpoints;
generating a VR representation of the identified object based on the received one or more images such that the generated VR representation includes visual 3D information of the identified object and at least a portion of the received one or more images is mapped to at least one viewing angle of the generated VR representation;
generating VR content corresponding to the original media content wherein the generated VR content includes the generated VR representation of the identified object and the generated VR representation is positioned within the VR content to correspond to a position of the identified object within the original media content; and
outputting the generated VR content as a VR version of the original media content, wherein a position of the generated VR representation of the identified object within the VR content is determined based on directional audio information associated with the identified object from the original media content.

2. The method of claim 1 , wherein:
the identified object is identified by matching an image of a frame of video of the original media content to image information associated with the identified object.

3. The method of claim 1 , wherein:
the identified object is identified by matching the audio from the original media content to audio information associated with the identified object.

4. The method of claim 1 , wherein:
the identified object is identified by matching the text from the original media content to text data associated with the identified object.

5. The method of claim 1 , further comprising:
determining an environment condition in the original media content; and
retrieving environment images corresponding to the determined environment condition,
wherein the retrieved environment images are incorporated in the generated VR content.

6. The method of claim 1 , further comprising:
identifying a scene in the original media content;
retrieving 3D scene information stored in a memory, the retrieved 3D scene information related to the original media content; and
generating the VR content by combining the retrieved 3D scene information and the original media content, wherein at least one feature of the retrieved 3D scene information is mapped to a corresponding at least one feature of the original media content.

7. The method of claim 1 , wherein the received one or more images includes at least one image corresponding to an object that is not depicted in the original media content.

8. A virtual reality (VR) content generating apparatus comprising:
a communication interface; and
at least one controller configured to:
identify an object in original media content;
generate metadata of the identified object;
search for images of the identified object using the generated metadata of the identified object;
receive one or more images depicting the identified object from a plurality of different viewpoints;
generate a VR representation of the identified object based on the received one or more images such that the generated VR representation includes visual 3D information of the identified object and at least a portion of the received one or more images is mapped to at least one viewing angle of the generated VR representation;
generate VR content corresponding to the original media content wherein the generated VR content includes the generated VR representation of the identified object and the generated VR representation is positioned within the VR content to correspond to a position of the identified object within the original media content; and
output the generated VR content as a VR version of the original media content,
wherein a position of the generated VR representation of the identified object within the VR content is determined based on directional audio information associated with the identified object from the original media content.

9. The apparatus of claim 8 , wherein:
the at least one controller is further configured to identify the identified object by matching an image of a frame of video of the original media content to image information associated with the identified object.

10. The apparatus of claim 8 , wherein:
the at least one controller is further configured to identify the identified object by comparing the audio from the original media content to audio information associated with the identified object.

11. The apparatus of claim 8 , wherein:
the at least one controller is further configured to identify the identified object by comparing the text from the original media content to text data associated with the identified object.

12. The apparatus of claim 8 , wherein the at least one controller is further configured to:
identify a scene in the original media content;
retrieve 3D scene information stored in a memory, the retrieved 3D scene information related to the original media content; and
generate the VR content by combining the retrieved 3D scene information and the original media content, wherein at least one feature of the retrieved 3D scene information is mapped to a corresponding at least one feature of the original media content.

13. The apparatus of claim 8 , wherein the received one or more images includes at least one image corresponding to an object that is not depicted in the original media content.

14. A machine-readable non-transitory medium having stored thereon machine-executable instructions for:
identifying an object in original media content;
generating metadata of the identified object;
searching for images of the identified object using the generated metadata of the identified object;
receiving one or more images depicting the identified object from a plurality of different viewpoints;
generating a VR representation of the identified object based on the received one or more images such that the generated VR representation includes visual 3D information of the identified object and at least a portion of the received one or more images is mapped to at least one viewing angle of the generated VR representation;
generating VR content corresponding to the original media content wherein the generated VR content includes the generated VR representation of the identified object and the generated VR representation is positioned within the VR content to correspond to a position of the identified object within the original media content; and
outputting the generated VR content as a VR version of the original media content,
wherein a position of the generated VR representation of the identified object within the VR content is determined based on directional audio information associated with the identified object from the original media content.

15. The machine-readable non-transitory medium of claim 14 , wherein the machine-executable instructions further include:
determining an environment condition in the original media content; and
retrieving environment images corresponding to the determined environment condition, wherein the retrieved environment images are incorporated in the generated VR content.

16. The machine-readable non-transitory medium of claim 14 , wherein the machine-executable instructions further include:
identifying a scene in the origina

Assignees

• Nbcuniversal Media Llc

Inventors

• Mason Thomas Bause

Classifications

• G06F16/954

  Navigation, e.g. using categorised browsing · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F16/00

  Information retrieval; Database structures therefor; File system structures therefor · CPC title

• G06F16/444

  Spatial browsing, e.g. 2D maps, 3D or virtual spaces · CPC title

• G06T7/50 Primary

  Depth or shape recovery · CPC title