Streaming the visible parts of a spherical video

Patent metadata

Field	Value
Publication number	US-2016112489-A1
Application number	US-201414518710-A
Country	US
Kind code	A1
Filing date	Oct 20, 2014
Priority date	Oct 20, 2014
Publication date	Apr 21, 2016
Grant date	—

Abstract

In a general aspect, a method includes determining a tile position in a frame of a spherical video based on a view perspective, selecting a portion of the frame of the spherical video as a two dimensional tile based on the tile position, encoding the two dimensional tile at a first quality, mapping the frame of the spherical video to a two dimensional representation of the spherical video based on a two dimensional projection algorithm, and encoding the two dimensional representation of the spherical video at a second quality.

First claim

What is claimed is:

1 . A method, comprising:
determining a tile position in a frame of a spherical video based on a view perspective; selecting a portion of the frame of the spherical video as a two dimensional tile based on the tile position; encoding the two dimensional tile at a first quality; mapping the frame of the spherical video to a two dimensional representation of the spherical video based on a two dimensional projection algorithm; and encoding the two dimensional representation of the spherical video at a second quality.

2 . The method of claim 1 , further comprising transmitting the encoded two dimensional tile and the encoded two dimensional representation as a streaming spherical video.

3 . The method of claim 1 , wherein the first quality is a higher quality as compared to the second quality.

4 . The method of claim 1 , wherein the view perspective is based on a viewable portion of the spherical video as seen by a viewer during a playback of the spherical video.

5 . The method of claim 1 , further comprising receiving an indication of the view perspective from a device executing a playback of the spherical video.

6 . The method of claim 1 , further comprising:
transmitting the encoded two dimensional tile over a first time period while streaming the spherical video; and transmitting the encoded two dimensional tile and the encoded two dimensional representation over a first time period while streaming the spherical video.

7 . The method of claim 1 , wherein the selecting of the portion of the frame of the spherical video as the two dimensional tile and the encoding of the two dimensional tile includes:
selecting the two dimensional tile from a datastore of previously encoded tiles; and reading the from the two dimensional tile from the datastore.

8 . The method of claim 1 , wherein the selecting of the portion of the frame of the spherical video as the two dimensional tile includes:
selecting the two dimensional tile from the frame of the spherical video as a frame to be encoded based on a position on the spherical video, wherein the position on the spherical video is based on the view perspective.

9 . The method of claim 1 , wherein the encoding of the two dimensional tile includes:
generating at least one residual for the two dimensional tile by subtracting a template from un-encoded pixels of a block of the two dimensional tile to be encoded; encoding the at least one residual by applying a transform to a residual block including the at least one residual; quantizing transform coefficients associated with the encoded at least one residual; and entropy encoding the quantized transform coefficients as at least one compressed video bit, wherein at least one of the generating of the at least one residual, the encoding of the at least one residual, the quantizing of the transform coefficients, and the quantizing of the transform coefficients includes setting of at least one parameter based on the first quality.

10 . The method of claim 1 , wherein the encoding of the two dimensional representation of the spherical video includes:
generating at least one residual for the two dimensional representation of the spherical video by subtracting a template from un-encoded pixels of a block of the two dimensional representation of the spherical video to be encoded; encoding the at least one residual by applying a transform to a residual block including the at least one residual; quantizing transform coefficients associated with the encoded at least one residual; and entropy encoding the quantized transform coefficients as at least one compressed video bit, wherein at least one of the generating of the at least one residual, the encoding of the at least one residual, the quantizing of the transform coefficients, and the quantizing of the transform coefficients includes setting of at least one parameter based on the second quality.

11 . A method, the method comprising:
receiving an encoded bit stream including an encoded two dimensional representation of a spherical video frame and an encoded two dimensional tile selected from the spherical video frame; decoding the two dimensional tile; decoding the two dimensional representation of the spherical video frame; converting the two dimensional representation to the spherical video frame; and replacing corresponding blocks of the spherical video frame with the decoded two dimensional tile.

12 . The method of claim 11 , wherein the receiving of the encoded bit stream includes receiving a header indicating a technique used during a conversion of a frame of the spherical video to the two dimensional representation of the spherical video frame.

13 . The method of claim 11 , wherein the replacing of the corresponding blocks of the spherical video frame with the decoded two dimensional tile includes one of pixel by pixel or block by block replacement of pixels or blocks in the decoded and converted spherical video frame with pixels or blocks of the decoded two dimensional tile.

14 . The method of claim 11 , wherein the converting of the two dimensional representation of the spherical video frame includes mapping the two dimensional representation of the spherical video frame to a spherical image using an inverse of a technique used to map the spherical video frame to the two dimensional representation of the spherical video frame.

15 . The method of claim 11 , further comprising:
generating a spherical video stream based on at least one spherical video frame including the replaced two dimensional tile such that during a playback of the spherical video stream, wherein a visible portion of the spherical video stream is of a higher quality than a non-visible portion of the spherical video stream.

16 . A non-transitory computer-readable storage medium having stored thereon computer executable program code which, when executed on a computer system, causes the computer system to perform steps comprising:
determining a tile position in a frame of a spherical video based on a view perspective; selecting a portion of the frame of the spherical video as a two dimensional tile based on the tile position; encoding the two dimensional tile at a first quality; mapping the frame of the spherical video to a two dimensional representation of the spherical video based on a two dimensional projection algorithm; and encoding the two dimensional representation of the spherical video at a second quality.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein the first quality is a higher quality as compared to the second quality.

18 . The non-transitory computer-readable storage medium of claim 16 , further comprising receiving an indication of the view perspective from a device executing a playback of the spherical video, wherein the view perspective is based on a viewable portion of the spherical video as seen by a viewer during the playback of the spherical video.

19 . The non-transitory computer-readable storage medium of claim 16 , wherein the selecting of the portion of the frame of the spherical video as the two dimensional tile and the encoding of the two dimensional tile includes:
selecting the two dimensional tile from a datastore of previously encoded tiles; and reading the two dimensional tile from the two dimensional tile from the datastore.

20 . The non-transitory computer-readable storage medium of claim 16 , wherein the selecting of the portion of the frame of the spherical

Assignees

• Google Inc

Inventors

• Adams Riley
• Bankoski Jim

Classifications

• H04N13/0055

  Electricity · mapped topic

• H04N13/0048

  Electricity · mapped topic

• H04L65/607 Primary

  Electricity · mapped topic

• H04N13/0059

  Electricity · mapped topic

• H04L65/765

  intermediate · CPC title