Generation of a sequence of textures for video delivery

Patent metadata

Field	Value
Publication number	US-11580675-B2
Application number	US-202117331186-A
Country	US
Kind code	B2
Filing date	May 26, 2021
Priority date	Mar 21, 2019
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

Techniques and systems are provided for generating a video from texture images, and for reconstructing the texture images from the video. For example, a texture image can be divided into a number of tiles, and the number of tiles can be sorted into a sequence of ordered tiles. The sequence of ordered tiles can be provided to a video coder for generating a coded video. The number of tiles can be encoded based on the sequence of ordered tiles. The encoded video including the encoded sequence of ordered tiles can be decoded. At least a portion of the decoded video can include the number of tiles sorted into a sequence of ordered tiles. A data file associated with at least the portion of the decoded video can be used to reconstruct the texture image using the tiles.

First claim

What is claimed is:

1. A method of reconstructing one or more texture images from a video, comprising:
obtaining at least a portion of decoded video comprising a number of segments based on a segment size parameter, the portion of decoded video including a first plurality of tiles sorted into a sequence of ordered tiles, wherein:
the segment size parameter is based on at least one of a receiving device constraint or a network condition,
an order of the sequence of tiles is based on the number of segments,
the first plurality of tiles are associated with a first texture image configured for application to a first three-dimensional mesh, and
each tile of the first plurality of tiles is temporally independent from other tiles of the first plurality of tiles;

obtaining a data file associated with at least the portion of the decoded video, the data file including contextual data mapping the first plurality of tiles to the first texture image; and
reconstructing the first texture image based on the contextual data mapping the first plurality of tiles to the first texture image.

2. The method of claim 1 , wherein contextual data for a tile of the first plurality of tiles includes at least a tile identifier, an identification of a texture image associated with the tile, and a location of the tile within the texture image.

3. The method of claim 2 , wherein the contextual data for the tile further includes an indication of a transform function, the transform function being configured to modify pixels of one or more tiles of the first plurality of tiles.

4. The method of claim 3 , further comprising:
applying an inverse transform function to the pixels of the one or more tiles of the first plurality of tiles, the inverse transform function being an inverse of the transform function.

5. The method of claim 1 , wherein the first plurality of tiles are sorted into the sequence of ordered tiles according to the segment size parameter in an order that minimizes a number of video segments needed to be downloaded to obtain the first texture image.

6. The method of claim 5 , wherein the sequence of ordered tiles includes a first sub-sequence and a second sub-sequence, the first sub-sequence including a first set of tiles from the first plurality of tiles and the second sub-sequence including a second set of tiles from the first plurality of tiles.

7. The method of claim 1 , wherein at least the portion of the decoded video further includes a second plurality of tiles, the second plurality of tiles being associated with a second texture image configured for application to at least one of the first three-dimensional mesh or a second three-dimensional mesh.

8. The method of claim 7 , wherein a first resolution of the first texture image and a second resolution of the second texture image are different resolutions, and wherein the first plurality of tiles and the second plurality of tiles have a uniform tile size.

9. The method of claim 1 , wherein a plurality of coded videos are generated for the sequence of ordered tiles, a first coded video of the plurality of coded videos having at least one of a different bit-rate, a different frame rate, or a different segment size than a second coded video of the plurality of coded videos, and further comprising:
receiving, over a network, at least one of a portion of the first coded video or a portion of the second coded video based on at least one or more network conditions associated with the network.

10. The method of claim 9 , wherein at least one of the portion of the first coded video or the portion of the second coded video is received further based on at least one of physical resources of the receiving device or an application of the receiving device.

11. A system comprising:
at least one processor and a memory storing instructions which, when executed by the at least one processor, cause the system to:
obtain at least a portion of decoded video comprising a number of segments based on a segment size parameter, the portion of decoded video including a first plurality of tiles sorted into a sequence of ordered tiles wherein:
the segment size parameter is based on at least one of a receiving device constraint or a network condition,
an order of the sequence of tiles is based on the number of segments,
the first plurality of tiles are associated with a first texture image configured for application to a first three-dimensional mesh, and
each tile of the first plurality of tiles is temporally independent from other tiles of the first plurality of tiles;
obtain a data file associated with at least the portion of the decoded video, the data file including contextual data mapping the first plurality of tiles to the first texture image; and
reconstruct the first texture image based on the contextual data mapping the first plurality of tiles to the first texture image.

12. The system of claim 11 , wherein at least the portion of the decoded video further includes a second plurality of tiles, the second plurality of tiles being associated with a second texture image configured for application to at least one of the first three-dimensional mesh or a second three-dimensional mesh.

13. The system of claim 12 , wherein a first resolution of the first texture image and a second resolution of the second texture image are different resolutions, and wherein the first plurality of tiles and the second plurality of tiles have a uniform tile size.

14. The system of claim 11 , wherein a plurality of coded videos are generated for the sequence of ordered tiles, a first coded video of the plurality of coded videos having at least one of a different bit-rate, a different frame rate, or a different segment size than a second coded video of the plurality of coded videos, and wherein the instructions, when executed, further cause the system to:
receive, over a network, at least one of a portion of the first coded video or a portion of the second coded video based on at least one or more network conditions associated with the network.

15. The system of claim 14 , wherein at least one of the portion of the first coded video or the portion of the second coded video is received further based on at least one of physical resources of the receiving device or an application of the receiving device.

16. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor, cause the processor to:
obtain at least a portion of decoded video comprising a number of segments based on a segment size parameter, the portion of decoded video including a first plurality of tiles sorted into a sequence of ordered tiles wherein:
the segment size parameter is based on at least one of a receiving device constraint or a network condition,
an order of the sequence of tiles is based on the number of segments,
the first plurality of tiles being associated with a first texture image configured for application to a first three-dimensional mesh, and
each tile of the first plurality of tiles is temporally independent from other tiles of the first plurality of tiles;

obtain a data file associated with at least the portion of the decoded video, the data file including contextual data mapping the first plurality of tiles to the first texture image; and
reconstruct the first texture image based on the contextual data mapping the first plurality of tiles to the first texture image.

17. The non-transitory computer readable storage medium of claim 16 , wherein at least the portion of the decoded video further includes a second plural

Assignees

• Adobe Inc

Inventors

• Simon Gwendal
• Swaminathan Viswanathan
• Carr Nathan
• Petrangeli Stefano

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• H04N19/182

  the unit being a pixel · CPC title

• H04N19/42 Primary

  characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) · CPC title

• H04N19/91

  Entropy coding, e.g. variable length coding [VLC] or arithmetic coding · CPC title

• H04N19/13

  Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC] · CPC title