Apparatus and method for parallel polyphase image interpolation

Patent metadata

Field	Value
Publication number	US-10332239-B2
Application number	US-201515557086-A
Country	US
Kind code	B2
Filing date	Jun 12, 2015
Priority date	Jun 12, 2015
Publication date	Jun 25, 2019
Grant date	Jun 25, 2019

Abstract

The present invention provides apparatus and method for parallel polyphase image interpolation. The apparatus comprises: a local memory, a first memory access control unit, a second memory access control unit, a source image data buffering unit, a filter coefficient buffering unit, a multiply-accumulator, a third memory access control unit and a state machine; the first memory access control unit is configured for obtaining source image data from the local memory and buffering them into a source image data buffering unit; the second memory access control unit is configured for obtaining filter coefficients from the local memory and buffering them into a filter coefficient buffering unit; the source image data buffering unit is configured for inputting source image data to each multiply-accumulator, the filter coefficient buffering unit is configured for broadcasting filter coefficients corresponding to the source image data to the multiply-accumulators; and the multiply-accumulator is configured for performing multiply-accumulation operation on the source image data and the filter coefficients, and outputting the multiply-accumulation result every F clocks. The interpolation apparatus of the present invention is reconfigurable and can support interpolation filters with any order, so it is universal.

First claim

What is claimed is:

1. An apparatus for parallel polyphase image interpolation, characterized by comprising:
a local memory for storing source image data and filter coefficients and destination image data;
a first memory access control unit for obtaining F rows with P pixels per row of source image data from the local memory and buffering the F rows of source image data into a source image data buffering unit;
a second memory access control unit for obtaining F filter coefficients from the local memory and buffering the F filter coefficients into a filter coefficient buffering unit;
the source image data buffering unit is configured for inputting P pixels of a f th row of source image data to P multiply-accumulators respectively in a k th clock cycle;
the filter coefficient buffering unit is configured for broadcasting a f th filter coefficient corresponding to the f th row of source image data to P multiply-accumulators in the k th clock cycle;
the multiply-accumulator is configured for performing multiply-accumulation operation on the f th row of source image data in the k th clock cycle and the f th filter coefficient and outputting the multiply-accumulation result and reset the accumulation register when f=F;
a third memory access control unit for obtaining the multiply-accumulation result and storing it into the local memory;
a state machine for outputting control signals to the first memory access control unit, the second memory access control unit, the third memory access control unit, the source image data buffering unit, and the filter coefficient buffering unit;
wherein, k 1, F 1, P>1, 1 f F.

2. The apparatus for parallel polyphase image interpolation according to claim 1 , characterized in that: the local memory is further configured for calculating coordinates of the image data to be interpolated to generate F rows of source image data, and for calculating all the filter coefficients needed for interpolation in local memory which form a interpolation filter coefficient table and storing them;
the first memory access control unit is further used for calculating the address of the source image data in the local memory, and reading the source image data;
the second memory access control unit is used for calculating the address of the F filter coefficients in the local memory and reading the filter coefficients.

3. The apparatus for parallel polyphase image interpolation according to claim 2 , characterized in that the local memory is further configured for calculating coordinates of the image data to be interpolated according to a simplest scaling ratio to generate F rows of source image data, and for calculating all the filter coefficients needed for interpolation in local memory which form a interpolation filter coefficient table and storing them, the interpolation filter coefficient table has L table entries and each entry corresponds to a filter with F coefficients. L is the output factor of a simplest scaling ratio, and F is the number of taps of each filter, and filters are arranged in an order of the interpolation time when the filter is used;
the first memory access control unit is further used for calculating the address of the source image data in the local memory according to a coordinate update table, and reading the source image data;
the second memory access control unit 1 is further used for calculating the address of the F filter coefficients in the local memory according to the interpolation time when the filter is used, and reading the filter coefficients.

4. The apparatus for parallel polyphase image interpolation according to claim 1 , characterized in that the local memory is a two-dimensional discrete memory.

5. The apparatus for parallel polyphase image interpolation according to claim 1 , characterized in that the first memory access control unit, the second memory access control unit or the third memory access control unit comprises at least one counter, at least one multiplier, at least one adder and at least one configuration register.

6. The apparatus for parallel polyphase image interpolation according to claim 1 , characterized in that the source image data buffering unit comprises at least one counter, at least one adder, at least one register bank and at least one configuration register.

7. The apparatus for parallel polyphase image interpolation according to claim 1 , characterized in that the filter coefficient buffering unit comprises at least one counter, at least one NOT gate and at least one register bank.

8. A method for parallel polyphase image interpolation, characterized by comprising:
obtaining F rows of source image data from the local memory and buffering the F rows of source image data into a source image data buffering unit by a first memory access control unit;
obtaining F filter coefficients from the local memory and buffering the F filter coefficients into a filter coefficient buffering unit by a second memory access control unit;
inputting, by the source image data buffering unit, P pixels of a f th row of source image data to P multiply-accumulators respectively in a k th clock cycle;
broadcasting, by the filter coefficient buffering unit, a f th filter coefficient corresponding to the f th row of source image data to P multiply-accumulators in the k th clock cycle;
performing, by the multiply-accumulator, multiply-accumulation operation on the f th row of source image data in the k th clock cycle and the f th filter coefficient, and outputting the multiply-accumulation result and reset the accumulation register when f=F;
obtaining, by the third memory access control unit, the multiply-accumulation result and storing it into the local memory;
determining if f+1 F, if yes, adding 1 to f and repeating the operation to the f th row until f+1>F, and obtaining the multiply-accumulation result of the k th clock cycle as an interpolation result of P destination image pixels;
wherein, k 1, F 1, 1 f F.
Repeat the said processing for P destination image pixels until all the destination image pixels are processed.

9. The method for parallel polyphase image interpolation according to claim 8 , characterized in that
before obtaining, by a first memory access control unit, F rows of source image data from the local memory and buttering the F rows of source image data into a source image data buffering unit, the method further comprises: calculating, by the local memory, coordinates of image data to be interpolated to generate F rows of source image data and calculating all the filter coefficients needed for interpolation in local memory which form a interpolation filter coefficient table and storing them.

10. The method for parallel polyphase image interpolation according to claim 8 , characterized in that
calculating, by the local memory, coordinates of image data to be interpolated to generate F rows of source image data and calculating all the filter coefficients needed for interpolation in local memory which form a interpolation filter coefficient table and storing them comprises:
calculating, by the local memory according to the simplest scaling ratio, coordinates of image data to be interpolated to generate F rows of source image data and calculating all the filter coefficients needed for interpolation in local memory which form a interpolation filter coefficient table and storing them.

Assignees

• Inst Automation Cas

Inventors

• Guo Ruoshan
• Wang Lei
• Han Rui
• Lin Chen
• Wang Donglin

Classifications

• G06T3/4053

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• G06T2210/52

  Parallel processing · CPC title

• G06T3/4007 Primary

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06T1/60

  Memory management · CPC title