Determining a region of interest on the basis of a HEVC-tiled video stream

Patent metadata

Field	Value
Publication number	US-10397666-B2
Application number	US-201515318631-A
Country	US
Kind code	B2
Filing date	Jun 26, 2015
Priority date	Jun 27, 2014
Publication date	Aug 27, 2019
Grant date	Aug 27, 2019

Abstract

A method is described for determining a region-of-interest (ROI) for a client device on the basis of at least one HEVC-tiled (panorama) video stream wherein the method comprises: receiving a ROI video stream for rendering a first ROI defining a first sub-region within the full image region of said HEVC-tiled video stream, the positions of HEVC tiles in said HEVC-tiled panorama video being defined by tile position information; providing ROI position information associated with at least one video frame of said ROI video stream, said ROI position information comprising at least first position of said first ROI; identifying one or more HEVC tiles of said HEVC-tiled video stream on the basis of said ROI position information and said tile position information, preferably one or more HEVC tiles that overlap with said first ROI; and, requesting video data associated with said one or more identified HEVC tiles for rendering a second ROI defining a second sub-region within the full image region of said HEVC-tiled video stream.

First claim

The invention claimed is:

1. Method for determining a region-of-interest (ROI) for a client device on the basis of at least one HEVC-tiled video stream, said HEVC-tiled video stream comprising a full image region, said method comprising:
receiving, at said client device, a ROI video stream for rendering a first ROI defining a first sub-region within said full image region of said HEVC-tiled video stream, the positions of HEVC tiles in said HEVC-tiled video stream being defined by tile position information;
providing, at said client device, ROI position information associated with at least one video frame of said ROI video stream, said ROI position information comprising at least a first position of said first ROI;
identifying, by said client device, one or more HEVC tiles of said HEVC-tiled video stream on the basis of said ROI position information and said tile position information; and,
requesting, by said client device, video data associated with said one or more identified HEVC tiles for rendering a second ROI defining a second sub-region within the full image region of said HEVC-tiled video stream;
wherein at least part of said ROI position information is transported in the bitstream of said ROI video stream to said client device as a SEI message or a ROI coordinates flag defining the location of said ROI coordinates in said bitstream; or,
wherein at least part of said ROI position information is transported in an elementary transport stream in an MPEG transport stream to the client device; or, as a DVB-defined SAD (Synchronized Auxiliary Data) packet in said MPEG transport stream, said SAD packet comprising a time-shifted ROI coordinate; or,
wherein at least part of said ROI position information is transported in a predefined box in an MPEG-4 stream, said predefined box comprising at least one of a moov, trak and/or stbl box of an MPEG-4 stream; or,
wherein at least part of said ROI position information is transported as a watermark to said client device in said ROI video stream.

2. Method according to claim 1 , wherein providing said ROI position information further comprises:
providing first ROI coordinates of at least a first image frame of said ROI video stream, said first ROI coordinates defining a first position of said first ROI at a first time instance of receiving an instruction for rendering a second ROI.

3. Method according to claim 2 wherein said instruction comprises a ROI vector for indicating a change of the position of said first ROI, said first ROI coordinates and said ROI vector being used for identifying said one or more HEVC tiles.

4. Method according to claim 2 wherein providing said ROI position information further comprises:
providing second ROI coordinates of at least a second image frame of said ROI video stream, said second ROI coordinates defining a second position of said first ROI at a second time instance of processing said ROI video stream by said client device; said first and second ROI coordinates being used for identifying said one or more HEVC tiles.

5. Method according to claim 1 wherein said requested video data are HEVC-encoded, said rendering of said second ROI comprising:
forming a decoded image region on the basis of said requested HEVC-encoded video data.

6. Method according to claim 5 wherein said rendering of said second ROI further comprises cropping said second ROI out of said decoded image region.

7. Method according to claim 1 wherein providing ROI position information further comprises:
providing look-ahead position information associated with at least one video frame of said ROI video stream that is not yet received by said client device, said look-ahead position information comprising a second position of a second ROI in said full image region.

8. Method according to claim 7 wherein said look-ahead position information comprises one or more time-shifted first ROI coordinates of one or more image frames in said ROI video stream that are not yet received by said client device.

9. Method according to claim 7 , wherein said time-shifted first ROI coordinates being provided to said client device prior to said one or more image frames.

10. Method according to claim 9 , wherein said time-shifted ROI coordinates being further associated with time-shift information for determining a time-shift.

11. Method according to claim 1
wherein at least part of said tile position information is provided to said client device in said HEVC-tiled video stream; and/or,
wherein at least part of said tile position information is provided in a spatial manifest file to said to said client device, said manifest file comprising one or more stream identifiers for locating one or more delivery nodes that are configured for delivering said HEVC-tiled video stream and/or said ROI video stream to said client device.

12. Method according to claim 11 , wherein at least part of said tile position information is provided to said client device in a moov or moof box in said HEVC-tiled video stream.

13. Method according to claim 1 wherein said video data associated with said one or more identified HEVC tiles are requested on the basis of a spatial manifest file comprising at least one or more HEVC tile identifiers and said tile position information.

14. A computer program product, implemented on a computer-readable non-transitory storage medium, comprising software code portions configured for, when run in the memory of a computer, executing the method steps according to claim 1 .

15. Method according to claim 1 , wherein identifying further comprises identifying one or more HEVC tiles that overlap said first ROI.

16. Method according to claim 1 , further comprising rendering, by said client device, said second ROI on the basis of said requested video data and said ROI position information.

17. Method according to claim 1 , wherein said video data associated with said one or more identified HEVC tiles are requested on the basis of a spatial manifest file comprising at least one or more HEVC tile identifiers, one or more URLs, and said tile position information; and, one or more ROI video stream identifiers associated with one or more ROI video streams, at least one of said one or more ROI video stream identifiers being associated with a dynamic indicator for indicating that the ROI position information changes in time.

18. Method according to claim 17 , wherein at least part of said ROI position information is transported in a predefined box in an MPEG-4 stream as a metadata track, the metadata track containing ROI coordinates as metadata with a specific sample entry code.

19. Method according to claim 1 , wherein at least part of said ROI position information is transported in a predefined box in an MPEG-4 stream as a metadata track, the metadata track containing ROI coordinates as metadata with a specific sample entry code.

20. Client device for rendering a region of interest (ROI), said client device being configured for:
receiving a ROI video stream for rendering a first ROI defining a first sub-region within the full image region of a HEVC-tiled video stream, the positions of HEVC tiles in said HEVC-tiled video stream being defined by tile position information;
providing ROI position information associated with at least one video frame of said ROI video stream, said ROI position information comprising at least a first position of said first ROI;
identifying one or more HEVC tiles of said HEVC-tiled video stream on the basis of said ROI position information and said tile position

Assignees

• Koninklijke Kpn Nv

Inventors

• Thomas Emmanuel
• Van Brandenburg Ray
• Van Deventer Mattijs Oskar

Classifications

• H04N21/234363

  by altering the spatial resolution, e.g. for clients with a lower screen resolution · CPC title

• H04N21/234327

  by decomposing into layers, e.g. base layer and one or more enhancement layers · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title

• H04N19/174

  the region being a slice, e.g. a line of blocks or a group of blocks · CPC title

• H04N19/167 Primary

  Position within a video image, e.g. region of interest [ROI] · CPC title