Video data processing method and apparatus

Patent metadata

Field	Value
Publication number	US-11563793-B2
Application number	US-202117372820-A
Country	US
Kind code	B2
Filing date	Jul 12, 2021
Priority date	Oct 10, 2016
Publication date	Jan 24, 2023
Grant date	Jan 24, 2023

Abstract

Example video data processing methods and apparatus are disclosed. One example method includes receiving a first stream from a client, where the first bitstream is obtained by encoding image data in a specified spatial object. The specified spatial object is part of panoramic space, and a size of the specified spatial object is larger than a size of a spatial object of the panoramic space corresponding to viewport information. The spatial object corresponding to the viewport information is located in the specified spatial object. The client receives a second stream, where the second bitstream is obtained by encoding image data of a panoramic image of the panoramic space with a lower resolution than a resolution of the image data included in the specified spatial object. The client plays the second bitstream and first bitstream.

First claim

What is claimed is:

1. A video data processing method, wherein the method comprises:
receiving, by a client, a first bitstream, wherein the first bitstream is obtained by encoding image data in a specified spatial object, wherein the specified spatial object is part of panoramic space, wherein a size of the specified spatial object is larger than a size of a spatial object of the panoramic space corresponding to viewport information, and wherein the spatial object corresponding to the viewport information is located in the specified spatial object;
receiving, by the client, a second bitstream, wherein the second bitstream is obtained by encoding image data of a panoramic image of the panoramic space with a lower resolution than a resolution of the image data included in the specified spatial object; and
playing, by the client, the second bitstream and the first bitstream.

2. The method according to claim 1 , wherein the second bitstream and the first bitstream are played based on a bitstream play mode, and the bitstream play mode comprises at least one of superposition play, supplementary play, or sampling play.

3. The method according to claim 1 , wherein the viewport information comprises location information of a viewport of a user in the panoramic space, and
wherein the location information comprises at least one of a center location, a start location, a left included angle and a right included angle of the center location, an upper included angle and a lower included angle of the center location, a length, or a width.

4. The method according to claim 1 , wherein the viewport information comprises a difference between location information of a viewport of a user at a current moment in the panoramic space and location information of a viewport of the user at a previous moment in the panoramic space; and
wherein the location information comprises at least one of a center location, a start location, a left included angle and a right included angle of the center location, an upper included angle and a lower included angle of the center location, a length, or a width.

5. The method according to claim 3 , wherein the viewport information further comprises displayed resolution information.

6. The method according to claim 2 , wherein the bitstream play mode is the superposition play; and
wherein the playing, by the client, the second bitstream and the first bitstream based on the bitstream play mode comprises:
decoding, by the client, the first bitstream and the second bitstream; and
performing superposition play on images of two decoded bitstreams.

7. The method according to claim 2 , wherein the bitstream play mode is the supplementary play; and
wherein the playing, by the client, the second bitstream and the first bitstream based on the bitstream play mode comprises:
decoding, by the client, the first bitstream and the second bitstream; and
supplementing an image of a decoded second bitstream to an image of a decoded first bitstream for play.

8. The method according to claim 2 , wherein the bitstream play mode is the sampling play; and
wherein the playing, by the client, the second bitstream and the first bitstream based on the bitstream play mode comprises:
decoding, by the client, the second bitstream;
sampling an image of a decoded second bitstream based on a resolution of an image corresponding to the first bitstream to obtain a sampled image; and
playing the sampled image.

9. The method according to claim 1 , wherein the playing, by the client, the second bitstream and the first bitstream comprises:
decoding, by the client, the second bitstream and the first bitstream based on an image data processing format supported by the client, and playing decoded images; or
decoding, by the client, the second bitstream and a part of the first bitstream based on an image data processing format supported by the client, and playing decoded images; or
decoding, by the client, a part of the second bitstream and the first bitstream based on an image data processing format supported by the client, and playing decoded images; or
decoding, by the client, a part of the second bitstream and a part of the first bitstream based on an image data processing format supported by the client, and playing decoded images,
wherein the image data processing format comprises at least one of an image data play format or an image data decoding format.

10. The method according to claim 1 , wherein the playing, by the client, the second bitstream and the first bitstream comprises:
selecting, by the client, a target bitstream from the second bitstream and the first bitstream based on an image data play mode selected by a user;
decoding the target bitstream; and
playing a decoded image based on the bitstream play mode.

11. The method according to claim 10 , wherein the method further comprises:
determining, based on the viewport information of the user at a current moment, a spatial object corresponding to a viewport of the user at the current moment; and
displaying the image data play mode at a specified display location of the spatial object corresponding to the viewport.

12. The method according to claim 10 , wherein the method further comprises:
determining, based on the viewport information of the user at a current moment, a spatial object corresponding to a viewport of the user at the current moment;
determining a location, in the panoramic space, of the spatial object corresponding to the viewport of the user at the current moment; and
displaying a schematic diagram of the location at a specified display location of the spatial object corresponding to the viewport of the user at the current moment.

13. A client device, comprising:
a receiver, at least one processor, and one or more memories;
wherein the one or more memories are coupled to the at least one processor and store programming instructions for execution by the at least one processor to:
receive, by the receiver, a first bitstream, wherein the first bitstream is obtained by encoding image data in a specified spatial object, wherein the specified spatial object is a part of panoramic space, wherein a size of the specified spatial object is larger than a size of a spatial object of the panoramic space corresponding to viewport information, and wherein the spatial object corresponding to the viewport information is located in the specified spatial object;
receive, by the receiver, a second bitstream, wherein the second bitstream is obtained by encoding image data of a panoramic image of panoramic space with a lower resolution than a resolution of the image data included in the specified spatial object; and
play the second bitstream and the first bitstream.

14. The client device according to claim 13 , wherein the second bitstream and the first bitstream are played based on a bitstream play mode, the bitstream play mode comprises at least one of superposition play, supplementary play, or sampling play.

15. The client device according to claim 13 , wherein the viewport information comprises location information of a viewport of a user in the panoramic space; and
wherein the location information comprises at least one of a center location, a start location, a left included angle and a right included angle of the center location, an upper included angle and a lower included angle of the center location, a length, or a width.

16. The client device according to claim 13 , wherein the viewport information comprises a difference between location information of a viewport of a user at a current moment in the panoramic space a

Assignees

• Huawei Tech Co Ltd

Inventors

• Xie Qingpeng
• Fu Jiali
• Di Peiyun

Classifications

• H04N13/00

  Stereoscopic video systems; Multi-view video systems; Details thereof · CPC title

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T15/20

  Perspective computation · CPC title

• H04L65/70

  Media network packetisation · CPC title

• G06T3/0087

  Physics · mapped topic