Methods and devices for texture coding

Patent metadata

Field	Value
Publication number	US-2025150606-A1
Application number	US-202418930384-A
Country	US
Kind code	A1
Filing date	Oct 29, 2024
Priority date	Nov 3, 2023
Publication date	May 8, 2025
Grant date	—

Abstract

A method for texture encoding, the method comprising the steps of: compressing a first colour channel of a given texture to obtain a plurality of fixed-sized data chunks representative of the first colour channel of a corresponding plurality of fixed-sized blocks of pixels of the given texture; for the given texture, training, using the plurality of fixed-sized data chunks, a neural network to output an inferred pixel colour block representative of at least a second colour channel of a respective fixed-sized block of pixels of the given texture based on an input representative of a respective fixed-sized data chunk representative of the first colour channel of the respective fixed-sized block of pixels of the given texture; and outputting at least one of the fixed-sized data chunks and the weights of the trained neural network as an encoded representation of at least a portion of the given texture.

First claim

1 . A method for texture encoding, the method comprising the steps of:
compressing a first colour channel of a given texture to obtain a plurality of fixed-sized data chunks representative of the first colour channel of a corresponding plurality of fixed-sized blocks of pixels of the given texture; for the given texture, training, using the plurality of fixed-sized data chunks, a neural network to output an inferred pixel colour block representative of at least a second colour channel of a respective fixed-sized block of pixels of the given texture based on an input representative of a respective fixed-sized data chunk representative of the first colour channel of the respective fixed-sized block of pixels of the given texture; and outputting at least one of the fixed-sized data chunks and weights of the trained neural network as an encoded representation of at least a portion of the given texture.

2 . The method according to claim 1 , in which
the step of training the neural network comprises a step of calculating a loss function for the neural network by using at least the second colour channel of the respective fixed-sized block of pixels of the given texture as a ground truth.

3 . The method according to claim 1 , in which
the neural network is trained using a mip of at least the second colour channel of the given texture at a mip level lower than a mip level of the given texture; and the step of outputting the encoded representation of at least a portion of the given texture comprises outputting at least a portion of the lower level mip corresponding to a region of the fixed-sized block of pixels of the given texture.

4 . A method for texture decoding, the method comprising the steps of:
storing a fixed-sized data chunk representative of a first colour channel of a fixed-sized block of pixels of a given texture; storing weights of a neural network trained, for the given texture, to output an inferred pixel colour block representative of at least a second colour channel of a respective fixed-sized block of pixels of the given texture based on an input representative of a respective fixed-sized data chunk representative of the first colour channel of the respective fixed-sized block of pixels of the given texture; decompressing the fixed-sized data chunk to obtain a first pixel colour block representative of the first colour channel of the fixed-sized block of pixels of the given texture; and inferring, using the trained neural network, a second pixel colour block representative of at least the second colour channel of the fixed-sized block of pixels of the given texture based on an input representative of the fixed-sized data chunk.

5 . The method according to claim 4 , further comprising a step of:
storing a mip of the at least the second colour channel of the given texture at a mip level lower than a mip level of the given texture, in which the neural network is trained using the lower level mip, in which the input to the trained neural network in the step of inferring is also representative of at least the second colour channel at a region of the lower level mip corresponding to a region of the fixed-sized block of pixels of the given texture.

6 . The method according to claim 1 , in which a respective fixed-sized data chunk is 6 bytes of data.

7 . The method according claim 1 , in which, a respective fixed-sized data chunk is 8 bytes of data.

8 . The method according to claim 1 , in which the first colour channel is one of the colour channels from a list consisting of:
(i) a green colour channel; (ii) a red colour channel; (iii) a blue colour channel; and (iv) a greyscale colour channel.

9 . A non-transitory computer-readable storage medium storing instructions which, when executed by a one or more processors, causes the one or more processors to perform operations comprising:
compressing a first colour channel of a given texture to obtain a plurality of fixed-sized data chunks representative of the first colour channel of a corresponding plurality of fixed-sized blocks of pixels of the given texture; for the given texture, training, using the plurality of fixed-sized data chunks, a neural network to output an inferred pixel colour block representative of at least a second colour channel of a respective fixed-sized block of pixels of the given texture based on an input representative of a respective fixed-sized data chunk representative of the first colour channel of the respective fixed-sized block of pixels of the given texture; and outputting at least one of the fixed-sized data chunks and weights of the trained neural network as an encoded representation of at least a portion of the given texture.

10 . The non-transitory computer-readable storage medium of claim 9 , the operations further comprising:
the step of training the neural network comprises a step of calculating a loss function for the neural network by using at least the second colour channel of the respective fixed-sized block of pixels of the given texture as a ground truth.

11 . The non-transitory computer-readable storage medium of claim 9 , the operations further comprising:
the neural network is trained using a mip of at least the second colour channel of the given texture at a mip level lower than a mip level of the given texture; and the step of outputting the encoded representation of at least a portion of the given texture comprises outputting at least a portion of the lower level mip corresponding to a region of the fixed-sized block of pixels of the given texture.

12 . The non-transitory computer-readable storage medium of claim 9 , the operations further comprising:
a respective fixed-sized data chunk is at least one of 6 bytes of data and 8 bytes of data.

13 . The non-transitory computer-readable storage medium of claim 9 , the operations further comprising:
the first colour channel is one of the colour channels from a list consisting of:
(i) a green colour channel;
(ii) a red colour channel;
(iii) a blue colour channel; and
(iv) a greyscale colour channel.

14 . A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform operations comprising:
storing a fixed-sized data chunk representative of a first colour channel of a fixed-sized block of pixels of a given texture; storing weights of a neural network trained, for the given texture, to output an inferred pixel colour block representative of at least a second colour channel of a respective fixed-sized block of pixels of the given texture based on an input representative of a respective fixed-sized data chunk representative of the first colour channel of the respective fixed-sized block of pixels of the given texture; decompressing the fixed-sized data chunk to obtain a first pixel colour block representative of the first colour channel of the fixed-sized block of pixels of the given texture; and inferring, using the trained neural network, a second pixel colour block representative of at least the second colour channel of the fixed-sized block of pixels of the given texture based on an input representative of the fixed-sized data chunk.

15 . The non-transitory computer-readable storage medium of claim 14 , the operations further comprising:
storing a mip of the at least the second colour channel of the given texture at a mip level lower than a mip level of the given texture, in which the neural network is trained using the lower level mip, in which the input to the trained neural network in the step of inferring is

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Cerrato Maurizio
• Leung Jun Yen
• Green Lawrence

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/40

  Analysis of texture (depth or shape recovery from texture G06T7/529) · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T15/04

  Texture mapping · CPC title