Systems and methods for changing a users perspective in virtual reality based on a user-selected position

Patent metadata

Field	Value
Publication number	US-11055914-B2
Application number	US-201916696596-A
Country	US
Kind code	B2
Filing date	Nov 26, 2019
Priority date	Dec 28, 2017
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

Systems and methods are described for a media guidance application (e.g., implemented on a user device) that allows users to select any arbitrary position in a virtual reality environment from where to view the virtual reality content and changes a user's perspective based on the selected position.

First claim

What is claimed is:

1. A method of generating for display a virtual reality media asset, the method comprising:
identifying an event occurring at a progression point in the virtual reality media asset;
identifying a friend associated with a user based on a user profile;
retrieving a video of a reaction of the friend to the event, wherein the video of the reaction of the friend to the event was captured via a camera during a first viewing experience of the virtual reality media asset which occurred at a first time;
generating a video object animating the reaction of the friend to the event based on the retrieved video that was captured by the camera during the first viewing experience of the virtual reality media asset;
inserting the video object into the virtual reality media asset at the progression point in response to identifying the event in the virtual reality media asset; and
generating for display to the user, during a second viewing experience occurring at a second time that is later than the first time at which the first viewing experience occurred, the virtual reality media asset including the video object inserted at the progression point.

2. The method of claim 1 , wherein identifying the event occurring at the progression point in the virtual reality media asset is based on metadata of the virtual reality media asset.

3. The method of claim 1 , further comprising determining placement of the video object in the virtual reality media asset by:
extracting depth information of the virtual reality media asset;
using segmentation to identify an attribute of the virtual reality media asset; and
superimposing depth information of the video object with the depth information of the virtual reality media asset.

4. The method of claim 3 , wherein the depth information of the video object is superimposed at a position in a point cloud of the virtual reality media asset that corresponds with the attribute of the virtual reality media asset and is closest to the virtual reality camera position.

5. The method of claim 1 , wherein the user profile is from one or more social media accounts of the user.

6. The method of claim 1 , wherein the video of the reaction of the friend is retrieved from a database of media assets of previous reactions captured by one or more facial expression capture devices.

7. The method of claim 1 , wherein the video of the reaction of the friend is retrieved from a social media account of the user or a social media account of the friend.

8. The method of claim 1 , wherein the video object is inserted in the virtual reality media asset at a location selected from:
beside a user position in the virtual reality media asset in response to the user viewing the virtual reality media asset from a perspective of the virtual reality camera position; and
near a location where the event occurs.

9. The method of claim 1 , further comprising:
generating for display, while the user is accessing the virtual reality media asset, content captured using multiple virtual reality cameras, wherein the virtual reality media asset is displayed on a virtual reality media asset display from a perspective of a virtual reality camera position of a virtual reality camera of the multiple virtual reality cameras and the virtual reality camera position is near an item of the content captured using the multiple virtual reality cameras.

10. The method of claim 1 , wherein identifying the event occurring at the progression point in the virtual reality media asset occurs while the user is accessing the virtual reality media asset, wherein the progression point is a time in the virtual reality media asset at which the event takes place.

11. A system for generating for display a virtual reality media asset, the system comprising:
control circuitry configured to:
identify an event occurring at a progression point in the virtual reality media asset;
identify a friend associated with a user based on a user profile;
retrieve a video of a reaction of the friend to the event, wherein the video of the reaction of the friend to the event was captured via a camera during a first viewing experience of the virtual reality media asset which occurred at a first time;
generate a video object animating the reaction of the friend to the event based on the retrieved video that was captured by the camera during the first viewing experience of the virtual reality media asset;
insert the video object into the virtual reality media asset at the progression point in response to identifying the event in the virtual reality media asset; and
generate for display to the user, during a second viewing experience occurring at a second time that is later than the first time at which the first viewing experience occurred, the virtual reality media asset including the video object inserted at the progression point.

12. The system of claim 11 , wherein the control circuitry is configured to identify the event occurring at the progression point in the virtual reality media asset based on metadata of the virtual reality media asset.

13. The system of claim 11 , wherein the control circuitry is further configured to determine placement of the video object in the virtual reality media asset by:
extracting depth information of the virtual reality media asset;
using segmentation to identify an attribute of the virtual reality media asset; and
superimposing depth information of the video object with the depth information of the virtual reality media asset.

14. The system of claim 13 , wherein the depth information of the video object is superimposed at a position in a point cloud of the virtual reality media asset that corresponds with the attribute of the virtual reality media asset and is closest to the virtual reality camera position.

15. The system of claim 11 , wherein the user profile is from one or more social media accounts of the user.

16. The system of claim 11 , wherein the video of the reaction of the friend is retrieved from a database of media assets of previous reactions captured by one or more facial expression capture devices.

17. The system of claim 11 , wherein the video of the reaction of the friend is retrieved from a user social media account or a friend social media account.

18. The system of claim 11 , wherein the video object is inserted in the virtual reality media asset at a location selected from:
beside a user position in the virtual reality media asset in response to the user viewing the virtual reality media asset from a perspective of the virtual reality camera position; and
near a location where the event occurs.

19. The system of claim 11 , wherein the control circuitry is further configured to:
generate for display, while the user is accessing the virtual reality media asset, content captured using multiple virtual reality cameras, wherein the virtual reality media asset is displayed on a virtual reality media asset display from a perspective of a virtual reality camera position of a virtual reality camera of the multiple virtual reality cameras and the virtual reality camera position is near an item of the content captured using the multiple virtual reality cameras.

20. The system of claim 11 , wherein the control circuitry is further configured to identify the event occurring at the progression point in the virtual reality media asset while the user is accessing the virtual reality media asset, wherein the progression point is a time in the virtual reality media asset at which the event takes place.

Assignees

• Rovi Guides Inc

Inventors

• Logan Jonathan A
• Bates Adam
• Jameela Hafiza
• Patterson Jesse F
• Berner Mark K
• Dorsey Eric
• Chamberlin David W
• Stevens Paul
• Waterman Herbert A

Classifications

• H04N21/21805 Primary

  enabling multiple viewpoints, e.g. using a plurality of cameras · CPC title

• G06T7/73

  using feature-based methods · CPC title

• H04N21/4524

  involving the geographical location of the client (retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00) · CPC title

• H04N21/4722

  for requesting additional data associated with the content · CPC title

• H04N21/47205

  for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally · CPC title