Image processing device for adjusting computational complexity of interpolation filter, image interpolation method, and image encoding method

Patent metadata

Field	Value
Publication number	US-2017070747-A1
Application number	US-201615176579-A
Country	US
Kind code	A1
Filing date	Jun 8, 2016
Priority date	Sep 9, 2015
Publication date	Mar 9, 2017
Grant date	—

Abstract

Provided are an image processing device capable of adjusting computational complexity, an image interpolation method, and an image encoding method. The image interpolation method includes selecting a horizontal interpolation filter having a first complexity, selecting a vertical interpolation filter having a second complexity, calculating pixel values of sub-pixels by performing an interpolation operation by using the selected horizontal and vertical interpolation filters, and generating interpolation information related to the selected horizontal interpolation filter and the selected vertical interpolation filter, wherein the first and second complexities are different from each other.

First claim

What is claimed is:

1 . An image interpolation method comprising:
selecting a horizontal interpolation filter having a first complexity setting; selecting a vertical interpolation filter having a second complexity setting; performing an interpolation operation on an input image using the selected horizontal and vertical interpolation filters; calculating pixel values of sub-pixels of the input image using results of the interpolation operation; and generating interpolation information regarding the input image related to the selected horizontal interpolation filter and the selected vertical interpolation filter, and
the first complexity setting and the second complexity setting are not equal.

2 . The image interpolation method of claim 1 , wherein
the interpolation operation includes performing horizontal and vertical interpolation on pixel values of a plurality of pixels of the input image, and generating a reference frame having the pixel values of sub-pixels to be used for a motion estimation operation based on the results of the interpolation operation; and the input image is a restored frame which is restored from an encoded original frame.

3 . The image interpolation method of claim 2 , further comprising:
determining at least one image characteristic of the restored frame, wherein the first complexity setting and the second complexity setting are selected according to the determined image characteristic.

4 . The image interpolation method of claim 3 , wherein
the determining of the at least one image characteristic of the restored frame includes determining a uniformity characteristic and an edge type of the restored frame.

5 . The image interpolation method of claim 1 , further comprising:
determining a hardware characteristic, the determining including
performing the interpolation operation by adjusting at least one of the first complexity setting and the second complexity setting, and
determining a degree of error of the interpolation operation; and
selecting the first complexity setting and the second complexity setting according to the determined hardware characteristic.

6 . The image interpolation method of claim 1 , wherein
the numbers of taps of the selected horizontal interpolation filter and the selected vertical interpolation filter are not equal.

7 . The image interpolation method of claim 1 , further comprising:
reading pixel values of a plurality of pixels of the input image, wherein the input image is a restored frame which is restored from an encoded original frame; performing a horizontal interpolation operation using the horizontal interpolation filter with respect to pixel values of M pixels arranged horizontally in the restored frame; and performing a vertical interpolation operation using the vertical interpolation filter with respect to pixel values of N pixels arranged vertically in the restored frame, where M and N are each an integer of one or more.

8 . The image interpolation method of claim 7 , further comprising:
selecting a vertical interpolation filter having a low complexity setting when the pixel values of N pixels are sequentially read during at least two read cycles; and selecting a horizontal interpolation filter having a low complexity setting when the pixel values of M pixels are sequentially read during at least two read cycles.

9 . The image interpolation method of claim 1 , wherein
the interpolation information includes first information indicating whether to adjust the first complexity setting and the second complexity setting, and second information indicating a first complexity setting value and a second complexity setting value.

10 . The image interpolation method of claim 9 , wherein
the first information and second information are included in header information of a bit stream.

11 . The image interpolation method of claim 10 , wherein
the header information includes a sequence parameter set (SPS) network abstraction layer (NAL) IP block and a picture parameter set (PPS) NAL IP block, and each of the first information and second information is included in any one of the SPS NAL IP block and the PPS NAL IP block.

12 . An image encoding method comprising:
selecting a horizontal interpolation filter and a vertical interpolation filter, the horizontal interpolation filter and the vertical interpolation filter each having a complexity setting, the respective complexity settings not being equal: performing an interpolation operation on an input image by using the selected horizontal interpolation filter and the selected vertical interpolation filter; performing motion estimation and compensation by using a reference frame related to the input image, the reference frame including pixel values of sub-pixels; and transmitting a bit stream including encoded image data and interpolation information related to the selected horizontal interpolation filter and the selected vertical interpolation filter.

13 . The image encoding method of claim 12 , wherein
the interpolation information includes information indicating a number of taps for the horizontal interpolation filter and the vertical interpolation.

14 . The image encoding method of claim 12 , wherein
the interpolation information includes first information indicating whether to adjust the complexity setting of the horizontal interpolation filter and the complexity setting of the vertical interpolation filter, and second information indicating the complexity settings of the horizontal interpolation filter and the vertical interpolation filter.

15 . The image encoding method of claim 12 , wherein
the interpolation information includes information for adjusting the complexity setting of the horizontal interpolation filter and the vertical interpolation filter by one frame unit, two or more frame units, or by block units, in one frame.

16 . An image decoding method comprising:
extracting, using at least one processor, encoded image data of a current frame of an image and information from a bit stream, the extracted information including information regarding at least one of a tap number setting, a bit depth setting, a filter coefficient setting, and a position setting to use with at least one interpolation filter; adjusting, using the at least one processor, a complexity setting of at least one interpolation filter based on the extracted information; interpolating, using the at least one processor, pixel values included in a restored frame using the at least one interpolation filter; and outputting, using the at least one processor, a reference frame including pixel values of sub-pixels.

17 . The method of claim 16 , further comprising:
performing at least one of motion estimation and compensation by using pixel values of the current frame and the reference frame.

18 . The method of claim 16 , wherein the adjusting the complexity setting includes:
selecting a complexity setting for a horizontal interpolation filter and a complexity setting for a vertical interpolation filter based on the information regarding at least one of the tap number setting, the bit depth setting, the filter coefficient setting, and the position setting.

19 . The method of claim 18 , wherein the selecting the complexity setting for the at least one interpolation filter includes selecting the complexity setting based on a desired amount of time required to complete an interpolation operation.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yang Jung-Yeop
• BYUN Ju-won

Classifications

• H04N19/587 Primary

  involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence · CPC title

• H04N19/65

  using error resilience · CPC title

• H04N19/52

  by predictive encoding · CPC title

• H04N19/44

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04N19/80

  Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence) · CPC title