Video streaming method

Patent metadata

Field	Value
Publication number	US-10600153-B2
Application number	US-201615057397-A
Country	US
Kind code	B2
Filing date	Mar 1, 2016
Priority date	Mar 5, 2015
Publication date	Mar 24, 2020
Grant date	Mar 24, 2020

Abstract

The invention relates to a video streaming method, comprising receiving a panoramic video; receiving head-tracking data; determining a current field of view based on the head-tracking data. The resolution of the panoramic video may be adjusted based on the current field of view, wherein resolution of the current field of view is a first resolution and the resolution outside of the current field of view is a second resolution. The panoramic video with adjusted resolution may be provided, wherein the first resolution is higher than the second resolution.

First claim

The invention claimed is:

1. A method comprising:
receiving a panoramic video;
receiving head-tracking data of a user of a head-mounted display;
determining a current field of view based on the head-tracking data:
determining a predicted field of view, that is expected to become visible;
generating three or more sub-streams by splitting the panoramic video into the three or more sub-streams, wherein at least one sub-stream resides in the current field of view, at least one sub-stream resides outside of the current field of view and the predicted field of view, and at least one sub-stream resides in the predicted field of view;
reducing resolution of the at least one sub-stream residing outside of the current field of view and the predicted field of view and reducing resolution of the at least one sub-stream residing in the predicted field of view; and
providing for streaming the panoramic video, the panoramic video having the reduced resolution of the at least one sub-stream residing outside of the current field of view and the predicted field of view, and the reduced resolution of the at least one sub-stream residing in the predicted field of view, wherein the resolution of the at least one sub-stream residing in the predicted field of view is lower than the resolution of the at least one sub-stream residing in the current field of view and is higher than the resolution of the at least one sub-stream residing outside of the current field of view and the predicted field of view.

2. The method according to claim 1 , further comprising:
determining a predicted field of view based on the head-tracking data;
adjusting resolution of the panoramic video based on the predicted field of view, wherein resolution of the predicted field of view is a third resolution; and
providing the panoramic video with the adjusted resolution based on the predicted field of view, wherein the third resolution is lower than the first resolution and higher than the second resolution.

3. The method according to claim 1 , further comprising:
determining at least one sub-stream residing in the predicted field of view; and
adjusting resolution of the at least one sub-stream residing in the predicted field of view, wherein the adjusted resolution of the at least one sub-stream residing in the predicted field of view is higher than the adjusted resolution of the at least one sub-stream residing outside of the current field of view and the predicted fields of view and lower than the at least one sub-stream residing in the current field of view.

4. The method according to claim 1 , further comprising:
determining a confidence of the head-tracking data; and
scaling the resolution of the panoramic video based on the confidence of the head-tracking data.

5. An apparatus comprising at least one processor, and memory including computer program code, the memory and the computer program code configured to, with at least one processor, cause the apparatus to perform at least the following:
receive a panoramic video;
receive head-tracking data of a user of a head-mounted display;
determine a current field of view based on the head-tracking data;
determine a predicted field of view, that is expected to become visible;
generate three or more sub-streams by splitting the panoramic video into the three or more sub-streams, wherein at least one sub-stream resides in the current field of view, at least one sub-stream resides outside of the current field of view and the predicted field of view, and at least one sub-stream resides in the predicted field of view;
reduce resolution of the at least one sub-stream residing outside of the current field of view and the predicted field of view and reduce resolution of the at least one sub-stream residing in the predicted field of view; and
provide for streaming the panoramic video, the panoramic video having the reduced resolution of the at least one sub-stream residing outside of the current field of view and the predicted field of view, and the reduced resolution of the at least one sub-stream residing in the predicted field of view, wherein the resolution of the at least one sub-stream residing in the predicted field of view is lower than the resolution of the at least one sub-stream residing in the current field of view and is higher than the resolution of the at least one sub-stream residing outside of the current field of view and the predicted field of view.

6. The apparatus according to claim 5 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
determine a predicted field of view based on head tracking data the head-tracking data;
adjust resolution of the panoramic video based on the predicted field of view, wherein resolution of the predicted field of view is a third resolution; and
provide the panoramic video with the adjusted resolution based on the predicted field of view, wherein the third resolution is lower than the first resolution and higher than the second resolution.

7. The apparatus according to claim 6 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
determine video stream latency information; and
control dimensions of the predicted field of view based on the video stream latency information.

8. The apparatus according to claim 5 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
determine at least one sub-stream residing in the predicted field of view; and
adjust resolution of the at least one sub-stream residing in the predicted field of view, wherein the adjusted resolution of the at least one sub-stream residing in the predicted field of view is higher than the adjusted resolution of the at least one sub-stream residing outside of the current field of view and lower than the at least one sub-stream residing in the current field of view.

9. The apparatus according to claim 5 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
determine a confidence of the head-tracking data; and
scale the resolution of the panoramic video based on the confidence of the head-tracking data.

10. The apparatus according to claim 9 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
receive gaze-tracking data in or along the head-tracking data; and
perform the scaling based on the gaze-tracking data.

11. The apparatus according to claim 5 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
stream the panoramic video into a mobile device.

12. The apparatus according to claim 5 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
receive gaze-tracking data;
determine a current focus area based on the gaze-tracking data;
adjust resolution of the panoramic video based on the current focus area, wherein resolution of the current focus area is higher than resolution outside of the current focus area; and
provide the panoramic video with the adjusted resolution based on the current focus area.

13. The apparatus according to claim 12 , further comprising computer program code, which executed by the at least one processor, causes the apparatus to perform:
determine a predicted focus area based on the gaze-tracking data;
adjust resolution of the panoramic video based on the predicted focus area; and
provide the panoramic vid

Assignees

• Nokia Technologies Oy

Inventors

• Roimela Kimmo
• You Yu

Classifications

• G09G2370/02

  Networking aspects · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• H04N21/8455

  involving pointers to the content, e.g. pointers to the I-frames of the video stream · CPC title

• H04N21/8456

  by decomposing the content in the time domain, e.g. in time segments · CPC title

• H04N21/4516

  involving client characteristics, e.g. Set-Top-Box type, software version or amount of memory available · CPC title