System and method for use in playing back panorama video content

Patent metadata

Field	Value
Publication number	US-2016012855-A1
Application number	US-201414340152-A
Country	US
Kind code	A1
Filing date	Jul 24, 2014
Priority date	Jul 14, 2014
Publication date	Jan 14, 2016
Grant date	—

Abstract

Some embodiments provide methods of playing back content, comprising: accessing video content comprising a series of frames that if fully decoded would extend beyond a viewer's field of view, and wherein each encoded frame comprises multiple encoded sections; determining a field of view of the viewer; identifying one or more sections of the first frame that are at least partially within the field of view; decoding the one or more sections of the first frame while not decoding one or more of the sections of the first frame that are not within the field of view; and displaying the one or more decoded sections of the first frame such that the portion of the first frame is displayed, and wherein less than all of the first frame is decoded and less than all of the first frame is displayed during playback.

First claim

What is claimed is:

1 . A method of playing back multimedia content, comprising:
accessing video content comprising a series of frames of visual content configured to be sequentially played back, wherein each frame of the series of frames if fully decoded would extend beyond a viewer's field of view, and wherein each encoded frame comprises multiple encoded sections; determining, prior to a portion of a first frame being displayed, a field of view of the viewer viewing the video content; identifying, as a function of the field of view, one or more sections of the first frame that are at least partially within the field of view and where at least a portion of the one or more sections are to be visible to the viewer upon displaying the portion of the first frame; decoding the one or more sections of the first frame that are at least partially within the field of view while not decoding one or more of the sections of the first frame that are not within the field of view; and displaying, according to playback timing of the video content and corresponding to when the first frame is to be displayed, the one or more decoded sections of the first frame such that the portion of the first frame is displayed, and wherein less than all of the first frame is decoded and less than all of the first frame is displayed during playback.

2 . The method of claim 1 , wherein the determining the field of view comprises determining that the position of the viewer's field of view is predicted to change to include at least a portion of one or more other sections of a subsequent frame yet to be displayed, wherein the one or more other sections correspond to one or more additional sections of the first frame that are not being displayed; and
decoding of at least the one or more other sections of the subsequent frame in response to the predicted change in the position of the field of view.

3 . The method of claim 2 , wherein the initiating the decoding of the at least the one or more other sections of the subsequent frame comprises initiating the decoding of one or more reference sections of one or more reference frames and upon which the subsequent frame references.

4 . The method of claim 2 , further comprising:
determining, in decoding the one or more other sections of one of the subsequent frames, that a first section that was decoded in a frame previous to a first of the subsequent frames is no longer within the field of view, and preventing the decoding of the first section while decoding the at least the one or more other sections of the first subsequent frame.

5 . The method of claim 1 , further comprising:
receiving orientation information; predicting, based on the orientation information and on a timing of when a portion of a subsequent frame is to be displayed, an orientation of the viewer's field of view; identifying, based on the predicted position of the viewer's field of view, one or more sections of the subsequent frame to be displayed that are anticipated to be at least partially within the predicted position of the viewer's field of view.

6 . The method of claim 1 , wherein the identifying the one or more sections of the first frame corresponding to the field of view comprises:
accessing a sections mapping, wherein the sections mapping comprises section position information of each of the multiple sections of the first frame and coordinate information across the multiple sections; and evaluating the sections mapping relative to the field of view.

7 . The method of claim 6 , wherein the evaluating the sections mapping comprises identifying coordinates of the coordinate information corresponding to the field of view, and identifying the one or more sections of the first frame that are at least partially within the identified coordinates corresponding to the field of view.

8 . The method of claim 7 , wherein the identifying the one or more sections of the first frame that are at least partially within the identified coordinates comprises identifying the one or more sections of the first frame that have at least a threshold amount of the section within the identified coordinates.

9 . The method of claim 1 , further comprising:
defining, for each of the multiple frames, the multiple sections of each frame such that the multiple sections extend across the entirety of a frame; and encoding each of the multiple sections such that each of the multiple sections is configured to be decoded independent of any of the other of the multiple sections of the frame.

10 . The method of claim 9 , further comprising:
defining, for each frame of the multiple frames, coordinate information extending over the entire frame; and specifying coordinates from the coordinate information for each of the sections of the frame.

11 . The method of claim 9 , further comprising:
identifying, for each section of the frame, neighboring sections and defining a mapping associating each section of the frame with each neighboring section.

12 . The method of claim 11 , wherein the defining the mapping comprises:
identifying boundaries of the frame; identifying two boundary sections of the multiple sections, wherein a first section of the two boundary sections has a boundary corresponding a first boundary of the frame, and a second section of the two boundary sections has boundary corresponding to a second boundary of the frame; and defining that the first boundary is adjacent to the second boundary defining a 360 degree frame.

13 . The method of claim 11 , further comprising:
identifying, as a function of the field of view, one or more other sections of the first frame that are not at least partially within the field of view; and decoding the one or more other sections of the first frame at a resolution lower than a resolution for which the one or more sections of the first frame that are at least partially within the field of view are decoded.

14 . A method of encoding video content, the method comprising:
accessing video content comprising multiple frames of visual content, wherein each of the multiple frames if fully displayed would extend beyond a field of view of a viewer; applying rectilinear projection to each frame of the multiple frames when the frame is in an equirectangular format; dividing each of the frames into a plurality of adjacent sections such that the plurality of adjacent sections of a frame extend over an entire area of the frame; and encoding, for each frame of the multiple frames, each of the plurality of sections such that each section of a frame is decodable independent of the other sections of the frame.

15 . The method of claim 14 , further comprising:
generating, for each of the multiple frames, a mapping of the sections, comprising:
defining, for each of the multiple frames, coordinate information extending over the entire frame; and
identifying coordinates from the coordinate information for each of the sections of the frame.

16 . The method of claim 15 , wherein the generating the mapping comprises identifying, for each section of the frame, neighboring sections and defining the mapping associating each section of the frame with each neighboring section.

17 . The method of claim 16 , wherein the generating the mapping comprises:
identifying boundaries of the frame; identifying two boundary sections of the multiple sections, wherein a first section of the two boundary sections has a boundary corresponding a first boundary of the frame, and a second section of the two boundary sections has boundary corresponding to a secon

Assignees

• Sony Computer Entertainment Inc

Inventors

• Krishnan Rathish

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N21/4728

  for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region · CPC title

• H04N5/2628

  Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation · CPC title

• H04N21/816

  involving special video data, e.g 3D video · CPC title

• G11B20/10

  Digital recording or reproducing · CPC title