Virtual linebuffers for image signal processors

Patent metadata

Field	Value
Publication number	US-10791284-B2
Application number	US-201916659702-A
Country	US
Kind code	B2
Filing date	Oct 22, 2019
Priority date	Jan 22, 2015
Publication date	Sep 29, 2020
Grant date	Sep 29, 2020

Abstract

In a general aspect, an apparatus can include image processing logic (IPL) configured to perform an image processing operation on pixel data corresponding with an image having a width of W pixels and a height of H pixels to produce output pixel data in vertical slices of K pixels using K vertically overlapping stencils of S×S pixels, K being greater than 1 and less than H, S being greater than or equal to 2, and W being greater than S. The apparatus can also include a linebuffer operationally coupled with the IPL, the linebuffer configured to buffer the pixel data for the IPL. The linebuffer can include a full-size buffer having a width of W and a height of (S−1). The linebuffer can also include a sliding buffer having a width of SB and a height of K, SB being greater than or equal to S and less than W.

First claim

What is claimed is:

1. A processor comprising:
image processing logic configured to perform a kernel function on each of multiple partially overlapping stencils, each stencil being a two-dimensional region of image data of an image frame; and
a virtual linebuffer comprising a sliding buffer and a fixed buffer, wherein the fixed buffer has a height that is less than a height of the stencil and has a width that is at least a width of the image frame, wherein the sliding buffer has a height that is at least as tall as two stencils that partially overlap vertically,
wherein the virtual linebuffer is configured to support consecutive image processing stages having different stencil sizes, the consecutive image processing stages including a producer kernel that writes to the virtual linebuffer and a consumer kernel that reads from the virtual linebuffer.

2. The processor of claim 1 , wherein the sliding buffer has a width that is at least as wide as two stencils that partially overlap horizontally and is less than a width of the image frame.

3. The processor of claim 1 , wherein a first stencil size of the producer kernel is different than a second stencil size of the consumer kernel, and
wherein the virtual linebuffer is configured to provide an input slice of image data to the consumer kernel based on multiple output slices of image data have been generated by the producer kernel.

4. The processor of claim 1 , wherein output slices generated by the producer kernel are taller and narrower than input slices consumed by the consumer kernel.

5. The processor of claim 1 , wherein output slices generated by the producer kernel are taller and wider than input slices consumed by the consumer kernel.

6. The processor of claim 1 , wherein output slices generated by the producer kernel has a height same to a height of input slices consumed by the consumer kernel, and has a width larger than a width of the input slices consumed by the consumer kernel.

7. The processor of claim 1 , wherein providing the multiple stencils that partially overlap horizontally and vertically comprises providing image data for four separate partially overlapping stencils to the image processing logic.

8. The processor of claim 1 , further comprising a consumer virtual linebuffer comprising a consumer sliding buffer and a consumer fixed buffer,
wherein the processor is configured to write a first portion of output of the kernel function to the consumer sliding buffer and a second portion of an output of the kernel function to the consumer fixed buffer.

9. The processor of claim 1 , wherein the virtual linebuffer is configured to perform, for each slice of pixels, operations comprising:
storing a first portion of a first slice of a first slice row in the fixed buffer;
storing a second portion of the first slice in the sliding buffer; and
moving data within the first slice including overwriting the first portion of the first slice in the fixed buffer with a portion of the second portion of the first slice in the sliding buffer at a location in the fixed buffer where the first portion was stored after the first slice has been read, the moved data being data that is reused between slice rows that are adjacently processed in a sequence, each slice row comprising a plurality of rows of image data.

10. The processor of claim 9 , wherein moving the data within the first slice from the sliding buffer to the fixed buffer comprises moving a portion of data equal to an amount of overlap between stencils in a stencil slice.

11. A method performed by a processor comprising: image processing logic configured to perform a kernel function on each of multiple partially overlapping stencils, each stencil being a two-dimensional region of image data of an image frame; and a virtual linebuffer comprising a sliding buffer and a fixed buffer, wherein the fixed buffer has a height that is less than a height of the stencil and has a width that is at least a width of the image frame, wherein the sliding buffer has a height that is at least as tall as two stencils that partially overlap vertically, the method comprising:
support, by the virtual linebuffer, consecutive image processing stages having different stencil sizes, the consecutive image processing stages including a producer kernel that writes to the virtual linebuffer and a consumer kernel that reads from the virtual linebuffer.

12. The method of claim 11 , wherein the sliding buffer has a width that is at least as wide as two stencils that partially overlap horizontally and is less than a width of the image frame.

13. The method of claim 11 , further comprising:
providing, by the virtual linebuffer, an input slice of image data to the consumer kernel based on multiple output slices of image data have been generated by the producer kernel,
wherein a first stencil size of the producer kernel is different than a second stencil size of the consumer kernel.

14. The method of claim 11 , wherein output slices generated by the producer kernel are taller and narrower than input slices consumed by the consumer kernel.

15. The method of claim 11 , wherein output slices generated by the producer kernel are taller and wider than input slices consumed by the consumer kernel.

16. The method of claim 11 , wherein output slices generated by the producer kernel has a height same to a height of input slices consumed by the consumer kernel, and has a width larger than a width of the input slices consumed by the consumer kernel.

17. The method of claim 11 , wherein providing the multiple stencils that partially overlap horizontally and vertically comprises providing image data for four separate partially overlapping stencils to the image processing logic.

18. The method of claim 11 , wherein the virtual linebuffer is a consumer virtual linebuffer having a consumer sliding buffer and a consumer fixed buffer, and further comprising writing a first portion of an output of the kernel function to the consumer sliding buffer and a second portion of an output of the kernel function to the consumer fixed buffer.

19. The method of claim 11 , further comprising performing, by the virtual linebuffer for each slice of pixels, operations comprising:
storing a first portion of a first slice of a first slice row in the fixed buffer;
storing a second portion of the first slice in the sliding buffer; and
moving data within the first slice including overwriting the first portion of the first slice in the fixed buffer with a portion of the second portion of the first slice in the sliding buffer at a location in the fixed buffer where the first portion was stored after the first slice has been read, the moved data being data that is reused between slice rows that are adjacently processed in a sequence, each slice row comprising a plurality of rows of image data.

20. The method of claim 19 , wherein moving the data within the first slice from the sliding buffer to the fixed buffer comprises moving a portion of data equal to an amount of overlap between stencils in a stencil slice.

Assignees

• Google Llc

Inventors

• Zhu Qiuling
• Shacham Ofer
• Redgrave Jason Rupert
• Finchelstein Daniel Frederic
• Meixner Albert

Classifications

• H04N5/262 Primary

  Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects {; Cameras specially adapted for the electronic generation of special effects} · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06T1/60 Primary

  Memory management · CPC title