Multi-view video processing method and apparatus

Patent metadata

Field	Value
Publication number	US-12015756-B2
Application number	US-202217752828-A
Country	US
Kind code	B2
Filing date	May 24, 2022
Priority date	Nov 29, 2019
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

Methods, apparatus, and systems for effectively reducing media content transmissions and efficiently rendering immersive media contents are disclosed. In one example aspect, a method includes requesting, by a user, media files from a server according to the current viewing position and viewing direction of the user, receiving, by the user, the media files from the server according to the current viewing position and the viewing direction of the user, extracting a patch of an atlas, and synthesizing the visual content in the current window area of the user, and obtaining, by the user, three-dimensional stereoscopic video content according to the current viewing position and viewing direction of the user.

First claim

What is claimed is:

1. A method of constructing media content, comprising:
placing a plurality of media samples into a media file associated with a plurality of views including one or more basic views and one or more additional views such that each media sample corresponds to one or more of the plurality of views and includes at least one of a texture component or a depth component of the corresponding view;
determining a basic view media track corresponding to the one or more basic views and one or more additional view media tracks such that each media track includes one or more indicators for describing information about the corresponding view; and
constructing the media content from the plurality of media samples based on the one or more indicators by grouping the plurality of media samples into one or more media sample groups each of which is associated with one basic view or by grouping a plurality of media tracks in which the plurality of media samples is placed,
wherein the media content is constructed using patches corresponding to one or more atlases of the one or more basic views and one or more atlases of the one or more additional views, wherein the one or more indicators include a first indicator for indicating the atlases of the one or more basic views, wherein the first indicator is located in the basic view media track, wherein the one or more atlases of the one or more basic views are determined by referring to the first indicator, and the one or more atlases of the one or more additional views are determined by referring to the one or more atlases of the one or more basic views.

2. The method of claim 1 , wherein each basic view corresponds to a basic view atlas.

3. The method of claim 1 , wherein the media content is constructed based on a combination of one or more basic views and one or more additional views.

4. The method of claim 1 , wherein the one or more indicators include an indicator for indicating whether each media track contains the texture component or the depth component or both the texture component and the depth component.

5. The method of claim 1 , wherein the one or more indicators include an atlas attribute indicator to define which part of the texture component and the depth component is contained in the media track.

6. The method of claim 1 , wherein the one or more indicators include a view identifier to describe the corresponding view.

7. The method of claim 1 , wherein the constructing of the media content includes combining patches from different media samples.

8. A method of constructing media content, comprising:
placing a plurality of media samples into a media file associated with a plurality of views including a plurality of basic views and a plurality of additional views such that each media sample corresponds to one of the plurality of views and includes at least one of a texture component or a depth component associated with the corresponding view;
determining a plurality of basic view media tracks corresponding to the plurality of basic views, respectively, and a plurality of additional view media tracks corresponding to the plurality of additional views, respectively, such that each media track includes one or more indicators for describing information about the corresponding view; and
constructing the media content from the plurality of media samples based on the one or more indicators by grouping the plurality of media samples into one or more media sample group each of which is associated with at least one basic view,
wherein the media content is constructed using patches corresponding to one or more atlases of the plurality of basic views and one or more atlases of the plurality of additional views, wherein the one or more indicators include a first indicator for indicating the atlases of the plurality of basic views, wherein the first indicator is located in the plurality of basic view media tracks, wherein the one or more atlases of the plurality of basic views are determined by referring to the first indicator, and the one or more atlases of the plurality of additional views are determined by referring to the one or more atlases of the plurality of basic views.

9. The method of claim 8 , wherein an image acquired based on one or more basic views is used as a base image for predicting other images.

10. The method of claim 9 , wherein each basic view corresponds to a basic view atlas.

11. The method of claim 10 , wherein the image is acquired based on the basic view atlas.

12. The method of claim 8 , wherein the one or more indicators include an entity level grouping indicator to describe a grouping type to group different media tracks containing different views.

13. The method of claim 8 , wherein each media track includes an indicator for identify the plurality of views as a basic view or an additional view.

14. A method of constructing media content, comprising:
placing, into a media file associated with a plurality of views, camera information including camera parameters corresponding to the plurality of views according to a viewing direction, a viewing position, and a viewing window;
selecting, based on the camera parameter information, media metadata from the media file; and
constructing the media content based on the media metadata,
wherein the media metadata is described by one or more media tracks including one or more basic view media tracks corresponding to one or more basic views and one or more additional view media tracks corresponding to one or more additional views, wherein the media content is constructed using patches corresponding to one or more atlases of the one or more basic views and one or more atlases of the one or more additional views, wherein the one or more basic view media tracks include a first indicator for indicating the atlases of the one or more basic views, wherein the one or more atlases of the one or more basic views are determined by referring to the first indicator, and the one or more atlases of the one or more additional views are determined by referring to the one or more atlases of the one or more basic views.

15. The method of claim 14 , wherein the camera information is extracted on a media file basis.

16. The method of claim 14 , wherein the camera information is extracted on a media track basis.

17. The method of claim 14 , wherein each of a plurality of media tracks is from a patch of the plurality of views, and wherein each view corresponds to one camera.

18. The method of claim 14 , wherein the plurality of views includes at least one basic view and at least one additional view associated with the at least one basic view.

19. The method of claim 14 , wherein the plurality of views includes two or more basic views stored in different media tracks, respectively.

20. The method of claim 14 , wherein the camera information includes a media track group to indicate that media data in the media track group is to be used to decode images within a certain space range corresponding to the media track group.

Assignees

• Zte Corp

Inventors

• Li Qiuting
• Huang Cheng

Classifications

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title

• H04N21/8153

  comprising still images, e.g. texture, background image · CPC title

• H04N21/44218

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title

• H04N21/21805 Primary

  enabling multiple viewpoints, e.g. using a plurality of cameras · CPC title