Virtual linebuffers for image signal processors

Patent metadata

Field	Value
Publication number	US-10277833-B2
Application number	US-201715479159-A
Country	US
Kind code	B2
Filing date	Apr 4, 2017
Priority date	Jan 22, 2015
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

In a general aspect, an apparatus can include image processing logic (IPL) configured to perform an image processing operation on pixel data corresponding with an image having a width of W pixels and a height of H pixels to produce output pixel data in vertical slices of K pixels using K vertically overlapping stencils of S×S pixels, K being greater than 1 and less than H, S being greater than or equal to 2, and W being greater than S. The apparatus can also include a linebuffer operationally coupled with the IPL, the linebuffer configured to buffer the pixel data for the IPL. The linebuffer can include a full-size buffer having a width of W and a height of (S−1). The linebuffer can also include a sliding buffer having a width of SB and a height of K, SB being greater than or equal to S and less than W.

First claim

What is claimed is:

1. A processor comprising:
a virtual linebuffer comprising a sliding buffer and a fixed buffer, wherein the sliding buffer has a height SH and a width SW, wherein the fixed buffer has a height FH and a width FW, wherein SH is greater than FH and FW is greater than SW, and wherein the virtual linebuffer is configured to:
store a first portion of a first slice of a first slice row in the fixed buffer,
store a second portion of the first slice in the sliding buffer, and
move data within the first slice including overwriting the first portion of the first slice in the fixed buffer with FH lines of the second portion of the first slice in the sliding buffer at a location in the fixed buffer where the first portion was stored after the first slice has been read, the moved data being data that is reused between slice rows that are adjacently processed in a sequence, each slice row comprising a plurality of rows of image data; and

image processing logic configured to perform a kernel function on each of multiple partially overlapping stencils of a slice of pixels using the first portion from the fixed buffer and the second portion from the sliding buffer, each stencil being a two-dimensional region of image data, and each slice of pixels being a portion of a slice row.

2. The processor of claim 1 , wherein the image processing logic is configured to reuse the FH lines of the second portion that was moved to the fixed buffer while processing a subsequent slice row in the sequence.

3. The processor of claim 1 , wherein at least one line of the second portion in the sliding buffer is not reused between slice rows that are adjacently processed in the sequence.

4. The processor of claim 1 , wherein the height FH of the fixed buffer is less than a height of each stencil, and wherein the height SH of the sliding buffer is greater than the height of each stencil.

5. The processor of claim 4 , wherein the height FH of the fixed buffer is defined to be the height of each stencil minus an amount of overlap between stencils in a slice of pixels.

6. The processor of claim 4 , wherein a sum of (i) the height SH of the sliding buffer and (ii) the height FH of the fixed buffer is greater than a size of two partially overlapping stencil regions.

7. The processor of claim 1 , wherein moving the FH lines of the second portion from the sliding buffer to the fixed buffer comprises moving a portion of data equal to an amount of overlap between stencils in a stencil slice.

8. The processor of claim 1 , further comprising a consumer virtual linebuffer comprising a consumer sliding buffer and a consumer fixed buffer,
wherein the processor is configured to write a first portion of output of the kernel function to the consumer sliding buffer and a second portion of an output of the kernel function to the consumer fixed buffer.

9. A method comprising:
processing, by a processor, a plurality of slice rows of image data in a sequence, each slice row comprising a plurality of rows of image data, including repeatedly:
storing a first portion of a first slice of a slice row in a fixed buffer and storing a second portion of the first slice in a sliding buffer, the fixed buffer and the sliding buffer together being a virtual linebuffer, wherein the sliding buffer has a height SH and a width SW, wherein the fixed buffer has a height FH and a width FW, wherein SH is greater than FH and FW is greater than SW;
loading, by the processor, a slice of pixels from the virtual linebuffer to memory of image processing logic, the slice of pixels comprising the first portion from the fixed buffer and the second portion from the sliding buffer;
performing, by the image processing logic, a kernel function on each of multiple partially overlapping stencils of the loaded slice of pixels using the first portion from the fixed buffer and the second portion from the sliding buffer, each stencil being a two-dimensional region of image data; and
after loading the first portion from the fixed buffer, moving data within the first slice including overwriting the first portion of the first slice in the fixed buffer with FH lines of the second portion of the first slice in the sliding buffer at a location in the fixed buffer where the first portion was stored.

10. The method of claim 9 , further comprising reusing the FH lines of the second portion moved to the fixed buffer when processing an adjacent slice row of the image data.

11. The method of claim 9 , wherein the height FH of the fixed buffer is less than a height of each stencil, and wherein the height SH of the sliding buffer is greater than the height of each stencil.

12. The method of claim 11 , wherein the height FH of the fixed buffer is defined to be the height of each stencil minus an amount of overlap between stencils in a slice of pixels.

13. The method of claim 11 , wherein a sum of (i) the height SH of the sliding buffer and (ii) the height FH of the fixed buffer is greater than a size of two partially overlapping stencil regions.

14. The method of claim 9 , wherein moving a portion of an output of the kernel function to the fixed buffer comprises moving a portion of data equal to an amount of overlap between stencils in a stencil slice.

15. The method of claim 9 , wherein the processor further comprises a consumer virtual linebuffer comprising a consumer sliding buffer and a consumer fixed buffer, the method further comprising:
writing a first portion of output of the kernel function to the consumer fixed buffer and a second portion of an output of the kernel function to the consumer sliding buffer.

Assignees

• Google Llc

Inventors

• Zhu Qiuling
• Shacham Ofer
• Redgrave Jason Rupert
• Finchelstein Daniel Frederic
• Meixner Albert

Classifications

• G06T1/60 Primary

  Memory management · CPC title

• H04N5/262 Primary

  Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects {; Cameras specially adapted for the electronic generation of special effects} · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title