Architecture for high performance, power efficient, programmable image processing

Patent metadata

Field	Value
Publication number	US-9965824-B2
Application number	US-201514694828-A
Country	US
Kind code	B2
Filing date	Apr 23, 2015
Priority date	Apr 23, 2015
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

An apparatus is described. The apparatus includes an image processing unit. The image processing unit includes a network. The image processing unit includes a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure to simultaneously process multiple overlapping stencils through execution of program code. The image processing unit includes a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network. The sheet generators are to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors. The image processing unit includes a plurality of line buffer units coupled to the network to pass line groups in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow.

First claim

The invention claimed is:

1. An apparatus, comprising:
an image processing unit, comprising:
a network;
a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure that supports bi-directional data movements along a horizontal axis and supports bi-directional data movements along a vertical axis, the array of execution unit lanes and the two-dimensional shift register to simultaneously process multiple overlapping stencils through execution of program code;
a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network, the sheet generators to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors;
a plurality of line buffer units coupled to the network to pass line groups of image data in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow.

2. The apparatus of claim 1 wherein the image processing unit is configurable to implement a DAG overall program flow.

3. The apparatus of claim 1 wherein the image processing unit is configurable to implement an image processing pipeline flow.

4. The apparatus of claim 1 wherein the image processing unit is configurable to cause a producing stencil processor to feed more than one consuming stencil processor.

5. The apparatus of claim 1 wherein the image processing unit is configurable to cause a consuming stencil processor to be fed by more than one producing stencil processor.

6. The apparatus of claim 1 wherein the image processing unit is configurable to simultaneously process different image streams with different stencil processors.

7. The apparatus of claim 1 wherein the array of execution unit lanes operate in SIMD fashion.

8. A non transitory machine readable storage medium containing program code that when processed by a computing system causes the computing system to simulate behavioral operation of an electronic circuit, said electronic circuit comprising:
an image processing unit, comprising:
a network;
a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure that supports bi-directional data movements along a horizontal axis and supports bi-directional data movements along a vertical axis, the array of execution unit lanes and the two-dimensional shift register to simultaneously process multiple overlapping stencils through execution of program code;
a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network, the sheet generators to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors;
a plurality of line buffer units coupled to the network to pass line groups of image data in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow.

9. The machine readable storage medium of claim 8 wherein the image processing unit is configurable to implement a DAG overall program flow.

10. The machine readable storage medium of claim 8 wherein the image processing unit is configurable to implement an image processing pipeline flow.

11. The machine readable storage medium of claim 8 wherein the image processing unit is configurable to cause a producing stencil processor to feed more than one consuming stencil processor.

12. The machine readable storage medium of claim 8 wherein the image processing unit is configurable to cause a consuming stencil processor to be fed by more than one producing stencil processor.

13. The machine readable storage medium of claim 8 wherein the image processing unit is configurable to simultaneously process different image streams with different stencil processors.

14. The machine readable storage medium of claim 8 wherein the array of execution unit lanes operate in SIMD fashion.

15. A computing system, comprising:
an image processing unit, comprising:
a network;
a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure that supports bi-directional data movements along a horizontal axis and supports bi-directional data movements along a vertical axis to simultaneously process multiple overlapping stencils through execution of program code;
a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network, the sheet generators to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors;
a plurality of line buffer units coupled to the network to pass line groups of image data in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow.

16. The computing system of claim 15 wherein the image processing unit is configurable to implement a DAG overall program flow.

17. The computing system of claim 15 wherein the image processing unit is configurable to implement an image processing pipeline flow.

18. The computing system of claim 15 wherein the image processing unit is configurable to cause a producing stencil processor to feed more than one consuming stencil processor.

19. The computing system of claim 15 wherein the image processing unit is configurable to cause a consuming stencil processor to be fed by more than one producing stencil processor.

20. The computing system of claim 15 wherein the image processing unit is configurable to simultaneously process different image streams with different stencil processors.

21. The computing system of claim 15 wherein the array of execution unit lanes operate in SIMD fashion.

Assignees

• Google Llc

Inventors

• Zhu Qiuling
• Shacham Ofer
• Meixner Albert
• Redgrave Jason Rupert
• Finchelstein Daniel Frederic
• Patterson David
• Desai Neeti
• Stark Donald
• Chang Edward T
• Mark William R

Classifications

• G06T1/60 Primary

  Memory management · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• H04N5/91

  Television signal processing therefor · CPC title

• H04N5/378

  Electricity · mapped topic

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title