Media information processing method and device, media information playback method and device, and storage medium

Patent metadata

Field	Value
Publication number	US-2025240464-A1
Application number	US-202318853883-A
Country	US
Kind code	A1
Filing date	May 15, 2023
Priority date	Jun 10, 2022
Publication date	Jul 24, 2025
Grant date	—

Abstract

A media information processing method and apparatus, a media information playback method and apparatus are disclosed. The media information processing method may include: acquiring media of a plurality of viewpoints, the plurality of viewpoints include at least one real viewpoint and at least one virtual viewpoint, and the media of the at least one virtual viewpoint is generated according to the media of the at least one real viewpoint; generating a media stream according to the media of the plurality of viewpoints, the media stream is a media file including media information; and performing segmentation and packing of the media stream, and generating a Media Presentation Description (MPD) text of the media, the MPD text includes a description of the real viewpoints and a description of the at least one virtual viewpoint, and the MPD text is used for acquiring media stream information of a viewpoint to be played.

First claim

1 . A media information processing method, comprising:
acquiring media of a plurality of viewpoints, wherein the plurality of viewpoints comprise at least one real viewpoint and at least one virtual viewpoint, and the media of the at least one virtual viewpoint is generated according to the media of the at least one real viewpoint; generating a media stream according to the media of the plurality of viewpoints, wherein the media stream is a media file comprising media information; and performing segmentation and packing of the media stream, and generating a Media Presentation Description (MPD) text of the media, wherein the MPD text comprises a description of the real viewpoints and a description of the at least one virtual viewpoint, and the MPD text is used for acquiring media stream information of a viewpoint to be played.

2 . The media information processing method of claim 1 , wherein the media stream comprises a plurality of real viewpoint media streams and a plurality of virtual viewpoint media streams; and
performing segmentation and packing of the media stream, and generating an MPD text of the media comprises: performing segmentation and packing of all the real viewpoint media streams to obtain a real viewpoint media segment file, wherein the real viewpoint media segment file comprises a plurality of real viewpoint media frames; performing segmentation and packing of all the virtual viewpoint media streams to obtain a virtual viewpoint media segment file, wherein the virtual viewpoint media segment file comprises a plurality of virtual viewpoint media frames; generating a real viewpoint media index file according to the real viewpoint media segment file, wherein the real viewpoint media index file comprises frame information of each of the real viewpoint media frames in the real viewpoint media segment file; generating a virtual viewpoint media index file according to the virtual viewpoint media segment file, wherein the virtual viewpoint media index file comprises frame information of each of the virtual viewpoint media frames in the virtual viewpoint media segment file; and generating the MPD text according to the real viewpoint media segment file, the virtual viewpoint media segment file, the real viewpoint media index file, and the virtual viewpoint media index file.

3 . The media information processing method of claim 2 , wherein performing segmentation and packing of all the real viewpoint media streams to obtain a real viewpoint media segment file comprises:
performing frame synchronization for all the real viewpoint media streams; merging all the frame-synchronized real viewpoint media streams into a single real viewpoint media stream; and performing segmentation and packing of the single real viewpoint media stream to obtain the real viewpoint media segment file.

4 . The media information processing method of claim 3 , wherein performing segmentation and packing of the single real viewpoint media stream to obtain the real viewpoint media segment file comprises:
performing segmentation and packing of the single real viewpoint media stream based on a Dynamic Adaptive Streaming over HTTP (DASH) protocol to obtain the real viewpoint media segment file.

5 . The media information processing method of claim 2 , wherein performing segmentation and packing of all the virtual viewpoint media streams to obtain a virtual viewpoint media segment file comprises:
performing frame synchronization for all the virtual viewpoint media streams; merging all the frame-synchronized virtual viewpoint media streams into a single virtual viewpoint media stream; and performing segmentation and packing of the single virtual viewpoint media stream to obtain the virtual viewpoint media segment file.

6 . The media information processing method of claim 5 , wherein performing segmentation and packing of the single virtual viewpoint media stream to obtain the virtual viewpoint media segment file comprises:
performing segmentation and packing of the single virtual viewpoint media stream based on a DASH protocol to obtain the virtual viewpoint media segment file.

7 . The media information processing method of claim 1 , wherein the MPD text comprises a MultiIdrIndex field, the MultiIdrIndex field is used for describing information of the real viewpoint media index file, and a format value of the real viewpoint media index file is an MPI type value;
wherein in response to the MultiIdrIndex field comprising an insert field, the MultiIdrIndex field is used for describing information of the virtual viewpoint media index file, and a value of the insert field represents a quantity of virtual viewpoints added between adjacent real viewpoints.

8 . (canceled)

9 . The media information processing method of claim 1 , wherein the MPD text comprises an AdaptationSet field; and in response to the AdaptationSet field comprising a cameras field, the AdaptationSet field is used for describing information of the real viewpoint media segment file, and the cameras field is used for representing a quantity of real viewpoints;
wherein in response to the AdaptationSet field comprising an insert field, the AdaptationSet field is used for describing information of the virtual viewpoint media segment file, and a value of the insert field represents a quantity of virtual viewpoints added between adjacent real viewpoints.

10 . (canceled)

11 . The media information processing method of claim 2 , wherein the real viewpoint media index file and the virtual viewpoint media index file are packed in a Moving Picture Experts Group Audio Layer IV (MP4) format, the real viewpoint media stream and the virtual viewpoint media stream each correspond to a MOOF box, and the MOOF box comprises a media frame size.

12 . The media information processing method of claim 1 , wherein the real viewpoint corresponds to a physical camera, the virtual viewpoint corresponds to a virtual camera, both the physical camera and the virtual camera are described by a camera descriptor, and the camera descriptor comprises at least one of:
camera indication information; position information of the camera; identification information of the camera; or identification information of the physical camera associated with the virtual camera.

13 . The media information processing method of claim 1 , wherein the real viewpoint corresponds to a physical camera, the virtual viewpoint corresponds to a virtual camera, both the real viewpoint and the virtual viewpoint are described by a free viewpoint descriptor, and the free viewpoint descriptor comprises at least one of:
identification information of the viewpoint; camera identification information corresponding to the viewpoint; camera indication information; or identification information of the physical camera associated with the virtual camera.

14 . The media information processing method of claim 2 , wherein the real viewpoint media frame and the virtual viewpoint media frame are both packed in an International Organization for Standardization Base Media File Format (ISO BMFF) media file.

15 . The media information processing method of claim 1 , wherein the media file is an ISO BMFF media file, the ISO BMFF media file comprises a free viewpoint information box, and the free viewpoint information box is used for describing viewpoint information in a media track or track fragment;
wherein the free viewpoint information box is used for indicating one or more free viewpoints comprised in a corresponding track and camera metadata information corresponding to the one or more free viewpoints; wherein the viewpoin

Assignees

• Zte Corp

Inventors

• Jiang Ping
• Li Qiuting

Classifications

• H04N21/84 Primary

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title

• H04N21/23412

  for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects · CPC title

• H04L65/60

  Network streaming of media packets · CPC title

• H04N21/23439

  for generating different versions · CPC title

• H04N21/26258

  for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40) · CPC title