Immersive video with advertisement content

Patent metadata

Field	Value
Publication number	US-11647258-B2
Application number	US-202017062310-A
Country	US
Kind code	B2
Filing date	Oct 2, 2020
Priority date	Jul 27, 2018
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

A system and method for inserting secondary content, e.g., advertisement content, graphics, images, etc., in a 360-degree immersive video environment. When a request is received from a client device for playing a video asset, a plurality of video tiles of the video asset are selected to be assembled as a video frame for delivery to the client device. A portion of the video tiles are identified that can be replaced with a corresponding set of advertisement content tiles, e.g., based on gaze vector information and/or a tile metadata specification containing advertisement insertion availability timing information with respect to each of the tiles of the video frame. After replacing the portion of the identified video tiles, the corresponding set of advertisement content tiles and remaining video tiles are assembled into the video frame including the advertisement content tiles at select locations, which is transmitted to the client device.

First claim

The invention claimed is:

1. A non-transitory machine-readable storage medium having program instructions thereon, which are configured to perform following acts when executed by a processor of a client device operative to engage in an immersive video session:
generating a request for playing an immersive video asset comprising a plurality of video frames, wherein each video frame comprises an array of tiles renderable for presentation in a display environment viewed by a user operating the client device;
providing gaze vector information relative to a video frame of the immersive video asset presented to the user in the display environment, the gaze vector information defining a gaze direction associated with a viewport in the display environment where the user is viewing at any particular time during the immersive video session, wherein the video frame comprises a plurality of regions viewable by the user and at least a first region within the viewport has a better video quality than at least a second region outside the viewport;
receiving at least one video frame of the immersive video asset, the at least one video frame including advertisement content at a location in the at least one video frame depending at least in part on the gaze vector information; and
rendering the at least one video frame including the advertisement content for presentation to the user.

2. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the gaze direction is determined by tracking at least one of an orientation of the user's headset associated with the client device for displaying the immersive video asset and a movement of the user's eyeballs with respect to different regions of the display environment while the immersive video asset is being displayed.

3. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are completely within the viewport.

4. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are completely outside the viewport.

5. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are partially within the viewport.

6. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are within a region of the at least one video frame having low activity.

7. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations of the at least one video frame depending further upon advertisement insertion availability timing information associated with the corresponding video asset tiles.

8. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the advertisement content comprises at least one of an advertisement still image and an advertisement video sequence.

9. The non-transitory machine-readable storage medium as recited in claim 1 , wherein the immersive video asset and the advertisement content are each tile-encoded into respective pluralities of data having different video qualities.

10. The non-transitory machine-readable storage medium as recited in claim 9 , wherein the respective pluralities of data corresponding to the immersive video asset and the advertisement content comprise respective sets of phase-encoded bitstreams.

11. The non-transitory machine-readable storage medium as recited in claim 9 , wherein the respective pluralities of data corresponding to the immersive video asset and the advertisement content comprise respective sets of block-intra-encoded bitstreams.

12. A client device configured to engage in an immersive video session, the client device comprising:
at least one microprocessor; and
a non-transitory computer-readable storage medium, coupled to the at least one microprocessor, further including computer-readable instructions, when executed by the at least one processor, configured for:
generating a request for playing an immersive video asset comprising a plurality of video frames, wherein each video frame comprises an array of tiles renderable for presentation in a display environment viewed by a user operating the client device;
providing gaze vector information relative to a video frame of the immersive video asset presented to the user in the display environment, the gaze vector information defining a gaze direction associated with a viewport in the display environment where the user is viewing at any particular time during the immersive video session, wherein the video frame comprises a plurality of regions viewable by the user and at least a first region within the viewport has a better video quality than at least a second region outside the viewport;
receiving at least one video frame of the immersive video asset, the at least one video frame including advertisement content at a location in the at least one video frame depending at least in part on the gaze vector information; and
rendering the at least one video frame including the advertisement content for presentation to the user.

13. The client device as recited in claim 12 , wherein the computer-readable instructions further comprise instructions operative for determining the gaze direction by tracking at least one of an orientation of the user's headset associated with the client device for displaying the immersive video asset and a movement of the user's eyeballs with respect to different regions of the display environment while the immersive video asset is being displayed.

14. The client device as recited in claim 12 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are completely within the viewport.

15. The client device as recited in claim 12 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are completely outside the viewport.

16. The client device as recited in claim 12 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are partially within the viewport.

17. The client device as recited in claim 12 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations that are within a region of the at least one video frame having low activity.

18. The client device as recited in claim 12 , wherein the advertisement content comprises one or more tiles that are multiplexed in to replace corresponding video asset tiles at respective locations of the at least one video frame depending further upon advertisement insertion availability timing information associated with the corresponding video asset tiles.

19. The client device as recited in c

Assignees

• Ericsson Telefon Ab L M

Inventors

• Phillips Chris
• Forsman Robert Hammond
• Changrani Vishal

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06Q30/0241

  Advertisements · CPC title

• H04N21/8153

  comprising still images, e.g. texture, background image · CPC title

• H04N21/26258

  for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40) · CPC title