Data processing method and apparatus, and device, storage medium and program product

Patent metadata

Field	Value
Publication number	US-12598333-B2
Application number	US-202318832063-A
Country	US
Kind code	B2
Filing date	Mar 1, 2023
Priority date	Mar 14, 2022
Publication date	Apr 7, 2026
Grant date	Apr 7, 2026

Abstract

A data processing method, apparatus and device, a storage medium, and a program product are disclosed. The method may include: acquiring free-viewpoint video data, the free-viewpoint video data comprises at least one texture map and/or at least one depth map; encapsulating the at least one texture map and/or the at least one depth map in at least one media track; and processing the free-viewpoint video data according to target-viewpoint information.

First claim

What is claimed is:

1 . A data processing method, comprising:
acquiring free-viewpoint video data, wherein the free-viewpoint video data comprises at least one texture map and/or at least one depth map; encapsulating the at least one texture map and/or the at least one depth map in at least one media track; and processing the free-viewpoint video data according to target-viewpoint information; wherein encapsulating the at least one texture map and/or the at least one depth map in at least one media track comprises: encapsulating the at least one texture map and/or the at least one depth map captured by a capturing apparatus in at least one media track, comprising:
encapsulating a texture map captured by the at least one capturing apparatus in at least one texture map media track; and
encapsulating a depth map captured by the at least one capturing apparatus in at least one depth map media track;
wherein:
a sample entry of the texture map media track comprises a first texture map box;
a sample entry of the depth map media track comprises a first depth map box;
the first texture map box comprises apparatus information of the capturing apparatus corresponding to the texture map media track or attribute information of the texture map; and
the first depth map box comprises apparatus information of the capturing apparatus corresponding to the depth map media track or attribute information of the depth map.

2 . The method of claim 1 , wherein encapsulating the at least one texture map and/or the at least one depth map captured by a capturing apparatus in at least one media track comprises:
encapsulating a texture map and/or a depth map captured by at least one capturing apparatus in a first media track.

3 . The method of claim 2 , wherein:
a sample of the first media track comprises at least one sub-sample; and the sub-sample comprises the texture map and/or the depth map.

4 . The method of claim 3 , wherein that the sub-sample comprises the texture map and/or the depth map comprises at least one of:
the sub-sample comprises texture maps and depth maps captured by a plurality of capturing apparatuses; or the sub-sample comprises a texture map and/or a depth map captured by one capturing apparatus.

5 . The method of claim 2 , wherein:
a sample entry of the first media track comprises a first box and/or a second box; the first box comprises apparatus information of the capturing apparatus; and the second box comprises attribute information of the texture map and/or the depth map.

6 . The method of claim 1 , wherein:
the sample entry of the texture map media track further comprises a second texture map box; the sample entry of the depth map media track further comprises a second depth map box; the second texture map box comprises apparatus information of the capturing apparatus corresponding to the texture map media track or attribute information of the texture map; the second depth map box comprises apparatus information of the capturing apparatus corresponding to the depth map media track or attribute information of the depth map; and information comprised in the first texture map box is different from information comprised in the second texture map box, and information comprised in the first depth map box is different from information comprised in the second depth map box.

7 . The method of claim 5 , wherein:
the first box further comprises track information; and the track information comprises at least one of: an identifier of a track where the texture map is located, or an identifier of a track where the depth map is located.

8 . The method of claim 1 , wherein:
the first texture map box further comprises an identifier of a track where the texture map is located; and the first depth map box further comprises an identifier of a track where the depth map is located.

9 . The method of claim 5 , wherein:
the apparatus information of the capturing apparatus comprises at least one of: a capturing apparatus quantity, a capturing apparatus identifier, or capturing apparatus parameter information; the attribute information of the texture map comprises at least one of: a texture map information identifier or texture map image information; and the attribute information of the depth map comprises at least one of: a depth map information identifier or depth map image information.

10 . The method of claim 1 , further comprising:
acquiring the target-viewpoint information, wherein acquiring free-viewpoint video data comprises: acquiring, according to the target-viewpoint information, free-viewpoint video data corresponding to the target view.

11 . The method of claim 10 , wherein:
acquiring, according to the target-viewpoint information, free-viewpoint video data corresponding to the target view comprises: acquiring a texture map and/or a depth map corresponding to the target view according to the target-viewpoint information.

12 . The method of claim 10 , wherein:
acquiring the target-viewpoint information comprises: acquiring the target-viewpoint information according to a timed metadata track, wherein the target view dynamically changes according to an indication of the timed metadata track.

13 . The method of claim 1 , wherein:
the target-viewpoint information comprises at least one of: a capturing apparatus identifier, capturing apparatus parameter information, target view position information, target view rotation information, or a target view identifier.

14 . The method of claim 12 , wherein:
the timed metadata track is associated with a media track where the free-viewpoint video data corresponding to the target view is located.

15 . The method of claim 1 , wherein processing the free-viewpoint video data according to target-viewpoint information comprises:
selecting, according to the free-viewpoint video data and the target-viewpoint information, a texture map and a depth map captured by a capturing apparatus corresponding to the target view, and synthesizing a texture map corresponding to the target view according to the texture map and the depth map; or selecting, according to the free-viewpoint video data and the target-viewpoint information, a texture map captured by a capturing apparatus corresponding to the target view.

16 . A non-transitory computer-readable storage medium, storing a computer-executable instruction which, when executed by a processor, causes the processor to carry out a data processing method, the method comprising:
acquiring free-viewpoint video data, wherein the free-viewpoint video data comprises at least one texture map and/or at least one depth map; encapsulating the at least one texture map and/or the at least one depth map in at least one media track; and processing the free-viewpoint video data according to target-viewpoint information; wherein encapsulating the at least one texture map and/or the at least one depth map in at least one media track comprises: encapsulating the at least one texture map and/or the at least one depth map captured by a capturing apparatus in at least one media track, comprising:
encapsulating a texture map captured by the at least one capturing apparatus in at least one texture map media track; and
encapsulating a depth map captured by the at least one capturing apparatus in at least one depth map media track;
wherein:
a sample entry of the texture map media track comprises a first texture map box;
a sample entry of the depth map media track comprises a first depth map box;
the first texture map box comprises apparatus informat

Assignees

• Zte Corp

Inventors

• Li Qiuting
• Huang Cheng
• Bai Yaxian
• Xie Shaowei

Classifications

• H04N21/23614

  Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28) · CPC title

• G06T15/20

  Perspective computation · CPC title

• G06T15/04

  Texture mapping · CPC title

• H04N21/816

  involving special video data, e.g 3D video · CPC title

• H04N21/21805 Primary

  enabling multiple viewpoints, e.g. using a plurality of cameras · CPC title