System and method for streaming visible portions of volumetric video

Patent metadata

Field	Value
Publication number	US-2021195165-A1
Application number	US-202117197276-A
Country	US
Kind code	A1
Filing date	Mar 10, 2021
Priority date	Aug 14, 2019
Publication date	Jun 24, 2021
Grant date	—

Abstract

Aspects of the subject disclosure may include, for example, a device having a processing system including a processor; and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising predicting a viewpoint within a volumetric video based on a movement input provided by a viewer, resulting in a predicted viewpoint at a future time, retrieving a cell occupancy bitmap for a point cloud of the volumetric video for the future time, determining visible cells based on the cell occupancy bitmap and the predicted viewpoint, where the visible cells are not obscured by points in other cells, and retrieving points of the point cloud that are within the visible cells prior to the future time.

First claim

What is claimed is:

1 . A device, comprising:
a processing system including a processor; and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising: predicting a viewpoint within a volumetric video based on a movement input provided by a viewer, resulting in a predicted viewpoint at a future time; retrieving a cell occupancy bitmap for a point cloud of the volumetric video for the future time; determining visible cells based on the cell occupancy bitmap and the predicted viewpoint, wherein the visible cells are not obscured by points in other cells; and retrieving points of the point cloud that are within the visible cells prior to the future time.

2 . The device of claim 1 , wherein the movement input comprises six degrees of freedom, and wherein the processing system comprises a plurality of processors operating in a distributed computing environment.

3 . The device of claim 1 , wherein the operations further comprise:
selecting a machine learning algorithm to predict the viewpoint; and predicting the viewpoint using the machine learning algorithm.

4 . The device of claim 3 , wherein the machine learning algorithm is a linear regression, a ridge regression, or a combination thereof.

5 . The device of claim 3 , wherein the machine learning algorithm is a deep learning scheme.

6 . The device of claim 5 , wherein the deep learning scheme is a Long Short Term Memory.

7 . The device of claim 1 , wherein the operations further comprise decompressing the cell occupancy bitmap.

8 . The device of claim 1 , wherein the operations further comprise retrieving additional cells for the predicted viewpoint.

9 . The device of claim 8 , wherein the operations further comprise:
merging the additional cells with the visible cells, resulting in merged cells; and rendering points of the point cloud that are within the merged cells.

10 . The device of claim 9 , wherein the operations further comprise retrieving a base layer for the additional cells.

11 . The device of claim 10 , wherein the operations further comprise retrieving enhancement layers for the additional cells responsive to available network bandwidth being sufficient.

12 . A non-transitory, machine-readable medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations, the operations comprising:
segmenting a point cloud of a volumetric video into three-dimensional cells; generating a cell occupancy bitmap for a frame of the volumetric video to be used at a future time; transmitting the cell occupancy bitmap for the frame, responsive to a first request received from a video player; and responsive to a second request received from the video player, transmitting cells to the video player for use at the future time, wherein the second request is based on at least some of the cells being visible to a viewer of the video player at a predicted viewpoint.

13 . The non-transitory, machine-readable medium of claim 12 , wherein the processing system comprises a plurality of processors operating in a distributed computing environment.

14 . The non-transitory, machine-readable medium of claim 12 , wherein the operations further comprise compressing the cell occupancy bitmap.

15 . The non-transitory, machine-readable medium of claim 12 , wherein the operations further comprise transmitting additional cells for the predicted viewpoint.

16 . The non-transitory, machine-readable medium of claim 15 , wherein the operations further comprise transmitting a base layer for the additional cells.

17 . The non-transitory, machine-readable medium of claim 16 , wherein the operations further comprise transmitting enhancement layers for the additional cells responsive to available network bandwidth being sufficient.

18 . A method, comprising:
predicting, by a processing system including a processor, a viewpoint within a volumetric video based on movement input by a viewer, resulting in a predicted viewpoint at a future time, wherein a point cloud of the volumetric video is segmented into three-dimensional cells; retrieving, by the processing system, a cell occupancy bitmap of the point cloud of the volumetric video for the future time; determining, by the processing system, visible cells based on the cell occupancy bitmap and the predicted viewpoint; and fetching, by the processing system, the visible cells prior to the future time.

19 . The method of claim 18 , further comprising rendering, by the processing system, points of the point cloud that are within the visible cells at the future time.

20 . The method of claim 18 , further comprising:
retrieving, by the processing system, additional cells for the predicted viewpoint; merging, by the processing system, the additional cells with the visible cells, resulting in merged cells; and rendering, by the processing system, points of the point cloud that are within the merged cells at the future time.

Assignees

• At & T Ip I Lp

Inventors

• Han Bo
• Pair Jackson Jarrell
• Ip Cheuk Yiu

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• H04N13/117

  the virtual viewpoint locations being selected by the viewers or determined by viewer tracking · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N20/00

  Machine learning · CPC title