Scene tracks for representing media assets

Patent metadata

Field	Value
Publication number	US-12573136-B2
Application number	US-202318356967-A
Country	US
Kind code	B2
Filing date	Jul 21, 2023
Priority date	Jul 21, 2022
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

Improved media processing techniques includes media data formats with a “convenience” sample having both a snapshot and a delta description of a scene at the same media time. In other examples, improved media processing techniques include creating a temporary media sample having a merged snapshot or a merged delta in a context of a pre-render thread to improve performance of rendering performed in a context of a separate rendering thread.

First claim

We claim:

1 . A non-transitory computer readable medium for representing a media asset comprising:
a sequence of entries for representing a model of objects in the asset, each entry describing the model over an interval of media time, the entries comprising: model snapshots each providing a self-contained definition of the model over its respective interval, and model deltas each providing a definition of the model over its respective interval with reference to a prior model defined in a prior entry, wherein the sequence of entries includes at least one convenience entry having a convenience model delta of the model deltas and a corresponding model snapshot of the model snapshots, and the model delta and the corresponding model snapshot describe a common interval of time.

2 . The medium of claim 1 , wherein a first model snapshot of the model snapshots for a first interval identifies objects in a scene, and a first model delta of the model deltas for a second interval identifies changes to the objects in the scene, new objects added to the scene, and/or removal of objects in the scene.

3 . The medium of claim 1 , wherein a first model delta of the model deltas for a first interval is defined with reference to another prior entry of a second model delta of the model deltas for a second interval.

4 . The medium of claim 1 , wherein the asset is a three-dimensional (3D) scene, the objects are 3D objects, and the model is a 3D model.

5 . A method for seeking within a media stream of media samples, comprising:
seeking within the media stream from a current media time to a desired media time including identifying a first media sample corresponding to the desired media time; when the first media sample contains a model delta at the desired media time, the model delta referring to a second media sample corresponding to a second media time: creating a temporary snapshot at the desired media time by merging the second media sample with the model delta at the desired media time; and rendering the temporary snapshot; and when the first media sample contains a snapshot at the desired media time, rendering the snapshot from the first media sample.

6 . The method of claim 5 , wherein the second media sample includes a snapshot at the second media time, the creating of the temporary snapshot includes:
merging the snapshot at the second media time with the model delta at the desired media time.

7 . The method of claim 5 , wherein the second media sample includes a model delta at the second media time that refers to a third media sample corresponding to a third media time, the creating of the temporary snapshot includes:
merging the model delta at the second media time with the model delta at the desired media time.

8 . The method of claim 7 , wherein the creating the temporary snapshot occurs on a computing device within a context of a first computer thread, and the rendering of the temporary snapshots occurs on the computing device within a context of a second computer thread, separate from the first computer thread.

9 . The method of claim 5 , wherein the media stream includes a sequence of entries for representing a model of objects in a media asset, each entry describing the model over an interval of media time, and the second media sample is a convivence media sample including:
a model snapshot at the desired media time providing a self-contained definition of the model over its respective interval including the desired media time, and a model deltas providing a definition of the model over the respective interval with reference to a model defined in a second media sample for a corresponding second media time.

10 . A method for seeking within a media stream of media samples, comprising:
rendering a current state of a scene by rendering a current media sample of the media stream corresponding to a current media time; seeking within the media stream from the current media time to a desired media time; identifying a sought media sample corresponding to the desired media time; when the sought media sample contains a model delta, creating a merged model delta by merging model deltas in media samples, wherein the model deltas corresponds to media times between the current media time and the desired media time; rendering the scene at the desired media time by applying the merged model delta to the current state.

11 . A non-transitory computer readable storage including instructions that, when executed by a processor, the processor causes:
seeking within a media stream from a current media time to a desired media time including identifying a first media sample corresponding to the desired media time; when the first media sample contains a model delta at the desired media time that refers to a second media sample corresponding to a second media time, creating a temporary snapshot at the desired media time by merging the second media sample with the model delta at the desired media time, and rendering the temporary snapshot; and when the first media sample contains a snapshot at the desired media time, rendering the snapshot from the first media sample.

12 . The computer readable storage of claim 11 , wherein the second media sample includes a snapshot at the second media time, and the creating of the temporary snapshot includes:
merging the snapshot at the second media time with the model delta at the desired media time.

13 . The computer readable storage of claim 11 , wherein the second media sample includes a model delta at the second media time that refers to a third media sample corresponding to a third media time, the creating of the temporary snapshot includes:
merging the model delta at the second media time with the model delta at the desired media time.

14 . The computer readable storage of claim 11 , wherein the creating the temporary snapshot occurs on a computing device within a context of a first computer thread, and the rendering of the temporary snapshots occurs on the computing device within a context of a second computer thread, separate from the first computer thread.

15 . The computer readable storage of claim 11 wherein the media stream includes a sequence of entries for representing a model of objects in a media asset, each entry describing the model over an interval of media time, and the second media sample is a convivence media sample including:
a model snapshot at the desired media time providing a self-contained definition of the model over its respective interval including the desired media time, and
a model deltas providing a definition of the model over the respective interval with reference to a model defined in a second media sample for a corresponding second media time.

16 . A rendering device, comprising:
a processor, and a computer readable storage including instructions that, when executed by the processor, the device causes:
seeking within a media stream from a current media time to a desired media time including identifying a first media sample corresponding to the desired media time;
when the first media sample contains a model delta at the desired media time that refers to a second media sample corresponding to a second media time, creating a temporary snapshot at the desired media time by merging the second media sample with the model delta at the desired media time, and rendering the temporary snapshot; and
when the first media sample contains a snapshot at the desired media time, rendering the snapshot from the first media sample.

17 . The rendering device of claim 16 , wherein the s

Assignees

• Apple Inc

Inventors

• Addey David J
• Angeli Alessandro
• Sahoo Shalini
• Flick Christopher L
• Bushell John Samuel
• Zhou Yi
• Idstrom Courtland M

Classifications

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2210/61

  Scene description · CPC title

• H04N21/85406

  involving a specific file format, e.g. MP4 format · CPC title

• H04N21/23412

  for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects · CPC title