Method and processor for streaming video processing

Patent metadata

Field	Value
Publication number	US-10848708-B2
Application number	US-201414901927-A
Country	US
Kind code	B2
Filing date	Jul 1, 2014
Priority date	Jul 1, 2013
Publication date	Nov 24, 2020
Grant date	Nov 24, 2020

Abstract

The invention provides methods of video processing in an efficient and/or low latency fashion and suitable for use in a streaming environment. Therefore the present invention relates to a system for video processing, a video image processor or a method of generating an output image by performing processing on an input image, wherein the processing is characterized in that it achieves the effect of a plurality of first unit image process steps, the method comprising the steps of performing sequentially one or a plurality of second image process filtering steps on said input image to generate said output image, wherein at least one of said second image process filtering steps achieves the effect of at least two, three, four, or five of said first image process steps.

First claim

The invention claimed is:

1. A method of video processing comprising:
receiving an input image including pixels and three color components; and
generating an output image by performing processing on the pixels of the input image, the processing on the pixels of the input image comprising:
for each of the pixels of the input image:
a first color component is received and written into a filter kernel buffer; and
a second color component and a third color component are alternatively received and written into the filter kernel buffer, such that either the second color component or the third color component are received and written into the filter kernel buffer;

performing a single combined filter operation for sequential execution of one or a plurality of image processing steps on the pixels of said input image to generate said output image;
wherein the filter kernel buffer is a pixel buffer having a fixed location per pixel and the second color component or the third color component missing for each of the pixels is identified as corrupt content in the filter kernel buffer;
wherein a particular image processing step of said one or a plurality of image processing steps simultaneously achieves the combined effect for a combination selected from:
vertical chroma up-sampling, de-interlacing, and vertical resizing;
de-interlacing and vertical scaling;
horizontal chroma-upsampling and horizontal scaling; or
vertical chroma-upsampling and vertical scaling;

wherein the particular image processing step is performed using one poly phase filter path per dimension, each polyphase filter path comprising:
a number of parallel convolution filters, one per color component; or
a single poly phase filter, one per color component.

2. The method of claim 1 , wherein: a second particular image processing step includes filtering that improves any one of output image quality, latency, execution cost of the method and/or the performance of the method, and
the particular image processing step achieves the combined effect of vertical chroma up-sampling, vertical resizing and de-interlacing, and
a third image processing step achieves the combined effect of chroma up-sampling, resizing, de-interlacing and packet loss reconstruction.

3. The method of claim 1 , wherein the particular image processing step achieves the combined effect for the combination of de-interlacing and vertical resizing by using an adaptive diagonal filtering method and/or based on methods only using linear operators.

4. The method of claim 1 , wherein the particular image processing step is performed using one poly phase filter path per dimension, each polyphase filter path comprising a number of parallel convolution filters, one per color component.

5. A video processor configured to generate an output image by performing processing on pixels of an input image having three color components, the processing on the pixels of the input image comprising:
performing a single combined filter operation for sequential execution of one or a plurality of image processing steps on the pixels of said input image to generate said output image;
wherein for each of the pixels of the input image:
a first color component is received and written into a filter kernel buffer; and
a second color component and a third color component are alternatively received and written into the filter kernel buffer, such that either the second color component or the third color component are received and written into the filter kernel buffer;

wherein the filter kernel buffer is a pixel buffer having a fixed location per pixel and the second color component or the third color component missing for each of the pixels is identified as corrupt content in the filter kernel buffer;
wherein a particular image processing step of said one or a plurality of image processing steps simultaneously achieves the combined effect for a combination selected from:
vertical chroma up-sampling, de-interlacing, and a vertical resizing;
de-interlacing and vertical scaling;
horizontal chroma-upsampling and horizontal scaling; or
vertical chroma-upsampling and vertical scaling;

wherein the particular image processing step is performed using one poly phase filter path per dimension, each polyphase filter path comprising:
a number of parallel convolution filters, one per color component; or
a single poly phase filter, one per color component.

6. The video processor of claim 5 , adapted so that said particular image processing step achieve the combined effect of vertical chroma upsampling, vertical resizing and de-interlacing and is implemented as two poly phase filter paths, each comprising a number of parallel convolution filters, one per color component or alternatively with a single poly phase filter, one per color component.

7. The video processor of claim 5 , adapted so that one of said image processing steps achieves the combined effect of chroma upsampling, resizing, de-interlacing and packet loss reconstruction, wherein equal kernel sizes are used for two or more or all of the underlying image processing steps.

8. The video processor of claim 5 , adapted so that one or more of said image processing steps is implemented as a filter, wherein at least part of the filter coefficients are stored in a Look Up Table.

9. The video processor of claim 5 , adapted so that:
one of said image process steps achieves the combined effect of chroma up-sampling and horizontal resizing; and/or
another of said image process steps achieves the combined effect of de-interlacing and vertical resizing.

10. The video processor of claim 9 , adapted so that the one of said image processing steps achieves the combined effect of chroma up-sampling and horizontal resizing and so that the one of said image processing steps is implemented as a number of parallel convolution filters, one per color component.

11. The video processor of claim 9 , adapted so that the another of said image processing steps achieves the combined effect of de-interlacing and vertical resizing and so that the another of said image processing steps is implemented with two parallel resizing filters or alternatively with a single motion detector modulated resizing filter.

12. The video processor of claim 5 , adapted so that one or more of said image process steps is selected as a convolution based filtering.

13. The video processor of claim 5 , adapted so that the particular image processing step achieves the combined effect for the combination of de-interlacing and vertical resizing by using an adaptive diagonal filtering method and/or based on methods only using linear operators.

14. The video processor of claim 5 , adapted so that one of the image processing steps includes achieving the effect of packet loss reconstruction using a packet reconstruction (filtering) method.

15. The video processor of claim 5 , wherein the processor comprises a processing engine and said one or more of said image process steps is adapted for execution on a processing engine.

16. The video processor of claim 5 , wherein the particular image processing step is performed using one poly phase filter path per dimension, each polyphase filter path comprising a number of parallel convolution filters, one per color component.

17. A controller for a video processor configured to generate an output image by performing processing on pixels of an input image having three color components, the processing on the pixels of the input image comprising:
performing a single combined filter operation for sequential execution of one or a plu

Assignees

• Barco Nv

Inventors

• Van Belle Ronny

Classifications

• G06T5/20

  using local operators · CPC title

• H04N7/0135

  involving interpolation processes (interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59) · CPC title

• G06T3/4007

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title

• H04N7/012 Primary

  Conversion between an interlaced and a progressive signal (for graphics images G09G2310/0229) · CPC title

• H04N5/21

  Circuitry for suppressing or minimising disturbance, e.g. moiré or halo · CPC title