Efficient convolution operations

Patent metadata

Field	Value
Publication number	US-11967012-B2
Application number	US-202217849322-A
Country	US
Kind code	B2
Filing date	Jun 24, 2022
Priority date	Jun 25, 2021
Publication date	Apr 23, 2024
Grant date	Apr 23, 2024

Abstract

A method of operation of a texturing/shading unit in a GPU pipeline is used for efficient convolution operations. The method uses texture hardware to collectively fetch all the texels required to calculate properties for a group of output pixels without any duplication. The method then bypasses bilinear filter hardware in the texture hardware and passes the fetched and unfiltered texel data from the texture hardware unit to shader hardware in the texturing/shading unit. The shader hardware uses the fetched texel data to perform a plurality of convolution operations to calculate the properties of each of the output pixel.

First claim

What is claimed is:

1. A method of operation of a texturing/shading unit in a GPU pipeline, the method comprising:
collectively fetching, by texture hardware in the texturing/shading unit, all texels required to calculate properties for a group of output pixels;
for all the texels required to calculate properties for the group of output pixels, bypassing bilinear filter hardware in the texture hardware and passing the fetched and unfiltered texel data from the texture hardware unit to shader hardware in the texturing/shading unit; and
performing a plurality of convolution operations in the shader hardware using the texel data to calculate the properties of each output pixel in the group of output pixels.

2. The method according to claim 1 , wherein each texel required to calculate properties for the group of output pixels is fetched only once.

3. The method according to claim 1 , further comprising, prior to performing the plurality of convolution operations:
broadcasting the texel data to registers allocated to instances corresponding to each of the output pixels.

4. The method according to claim 3 , wherein all the fetched texel data for a group of output pixels is broadcast to registers allocated to each of the instances.

5. The method according to claim 3 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:
accessing texel data stored in registers allocated to corresponding instance; and
performing a plurality of convolution operations using the accessed texel data and a set of weights to calculate the properties for the output pixel, wherein the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero.

6. The method according to claim 3 , wherein the fetched texel data is selectively broadcast to registers allocated to instances corresponding to each of the output pixels such that registers for an instance only receive texel data for use in performing convolution operations for that instance.

7. The method according to claim 3 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:
accessing texel data stored in registers allocated to corresponding instance; and
performing a plurality of convolution operations using the accessed texel data to calculate the properties for the output pixel.

8. The method according to claim 1 , wherein the fetched texel data is stored in shared registers for the group of output pixels.

9. The method according to claim 8 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:
accessing a subset of the fetched texel data for the group of output pixels from the shared registers, the subset corresponding to a kernel of the convolution for the output pixel with zeros; and
performing a convolution operation in the shader hardware using the accessed texel data to calculate the properties for the output pixel.

10. The method according to claim 8 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:
accessing a weight for each texel in the fetched texel data, wherein the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero; and
performing a convolution operation in the shader hardware using the fetched texel data and the accessed weights to calculate the properties for the output pixel.

11. The method according to claim 10 , wherein accessing a weight for each texel in the fetched texel data comprises selecting a subset of a weight array, the subset dependent upon the output pixel and wherein in the selected subset the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero.

12. The method according to claim 10 , wherein accessing a weight for each texel in the fetched texel data comprises accessing one of a plurality of pre-defined weight arrays, each of the weight arrays corresponding to a different one of the plurality of output pixels and in which a different subset of the weights are set to zero.

13. The method according to claim 1 , wherein collectively fetching all texels required to calculate properties for a group of output pixels comprises:
executing a plurality of fetch operations for different sample positions, each sample position being offset from a texel centre and defining a different, non-overlapping patch of adjacent texels to be fetched.

14. The method according to claim 1 , wherein collectively fetching all texels required to calculate properties for a group of output pixels comprises:
executing a plurality of gather operations for different sample positions, each sample position being offset by integer values from a centre of one of the output pixels and defining a different, non-overlapping patch of adjacent texels to be fetched.

15. A texturing/shading unit for use in a GPU pipeline, the texturing/shading unit comprising:
texture hardware comprising a fetch unit and bilinear filter hardware; and
shader hardware,
wherein the texture hardware is arranged to fetch, in the fetch unit, all texels required to calculate properties for a group of output pixels,
for all the texels required to calculate properties for the group of output pixels, bypass the bilinear filter hardware and output the fetched and unfiltered texel data to shader hardware, and
the shader hardware is arranged to perform a plurality of convolution operations using the texel data to calculate the properties of each output pixel in the group of output pixels.

16. The texturing/shading unit according to claim 15 , wherein each texel required to calculate properties for the group of output pixels is fetched only once.

17. The texturing/shading unit according to claim 15 , wherein the shader hardware is arranged, prior to performing the plurality of convolution operations, to broadcast the texel data to registers allocated to instances corresponding to each of the output pixels.

18. The texturing/shading unit according to claim 17 , wherein all the fetched texel data for a group of output pixels is broadcast to registers allocated to each of the instances.

19. The texturing/shading unit according to claim 17 , wherein the shader hardware is arranged to perform the plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel, by:
accessing texel data stored in registers allocated to corresponding instance; and
performing a plurality of convolution operations using the accessed texel data and a set of weights to calculate the properties for the output pixel, wherein the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero.

20. A non-transitory computer readable storage medium having stored thereon a computer readable dataset description of an integrated circuit that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture a texturing/shading unit for use in a GPU pipeline, the texturing/shading unit comprising:
texture hardware comprising a fetch unit and a bilinear filter hardware; an

Assignees

• Imagination Tech Ltd

Inventors

• King Rostam
• Thomas William

Classifications

• G06T15/005 Primary

  General purpose rendering architectures · CPC title

• G06T15/04 Primary

  Texture mapping · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06T2200/28

  involving image processing hardware · CPC title