Image processing apparatus and method

Patent metadata

Field	Value
Publication number	US-9277235-B2
Application number	US-201414341319-A
Country	US
Kind code	B2
Filing date	Jul 25, 2014
Priority date	Feb 19, 2009
Publication date	Mar 1, 2016
Grant date	Mar 1, 2016

Abstract

The present invention relates to an image processing apparatus and method that can improve encoding efficiency while preventing an increase in load.
An extraction circuit 71 of a filtering prediction circuit 64 extracts motion compensation images for generating a prediction image in a high-resolution enhancement layer from reference frames in a low-resolution base layer. A filtering circuit 72 of the filtering prediction circuit 64 performs filtering, which involves upconversion and which uses analysis in the time direction, on a plurality of motion compensation images in the base layer extracted by the extraction circuit 71 to generate a prediction image in the enhancement layer. The present invention can be applied to, for example, an encoding apparatus and a decoding apparatus.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
processing circuitry including:
an extraction circuitry portion that performs motion compensation using, as reference frames, frames formed of decoded images and using motion vectors in images that have been encoded, and that extracts a plurality of motion compensation images having a lower resolution than a prediction image from the reference frames corresponding to the prediction image; and
a prediction imaging circuitry portion that performs a filtering process to compensate the plurality of motion compensation images extracted by the extraction circuitry portion for high-frequency components by utilizing correlation in a time direction that is included in the plurality of motion compensation images, thereby generating the prediction image that has a higher resolution than the plurality of motion compensation images,
wherein the prediction imaging circuitry portion includes:
a converter that converts a resolution of a difference image between the plurality of motion compensation images extracted by the extraction circuitry portion and that increases the resolution of the difference image,
a first filter that applies a low-pass filter to the difference image whose resolution has been increased by the converter,
a second filter that applies a high-pass filter to an image obtained by applying the low-pass filter by the first filter, and
an adder that adds the image obtained by applying the low-pass filter by the first filter and another image obtained by applying the high-pass filter by the second filter to one of the plurality of motion compensation images extracted by the extraction circuitry portion, and that generates the prediction image.

2. The image processing apparatus according to claim 1 ,
wherein the one of the extracted plurality of motion compensation images is a motion compensation image extracted from a preceding frame with respect to a time of the prediction image.

3. An image processing method comprising:
performing motion compensation using, as reference frames, frames formed of decoded images and using motion vectors in images that have been encoded, and extracting a plurality of motion compensation images having a lower resolution than a prediction image from the reference frames corresponding to the prediction image; and
performing a filtering process to compensate the extracted plurality of motion compensation images for high-frequency components by utilizing correlation in a time direction that is included in the plurality of motion compensation images, thereby generating the prediction image that has a higher resolution than the plurality of motion compensation images,
wherein said performing the filtering process includes:
converting a resolution of a difference image between the extracted plurality of motion compensation images and increasing the resolution of the difference image,
applying a low-pass filter to the difference image whose resolution has been increased,
applying a high-pass filter to an image obtained by said applying the low-pass filter,
adding the image obtained by said applying the low-pass filter and another image obtained by said applying the high-pass filter to one of the extracted plurality of motion compensation images, and
generating the prediction image.

4. An image processing apparatus comprising:
processing circuitry including:
a detecting circuitry portion that detects motion vectors based on images and an original image that is an image to be encoded, each of the images being obtained by performing local decoding based on a residual signal indicating a difference between the original image and a prediction image;
an extraction circuitry portion that performs motion compensation using, as reference frames, frames formed of the locally decoded images and using the motion vectors detected by the detecting circuitry portion, and that extracts a plurality of motion compensation images having a lower resolution than the prediction image from the reference frames corresponding to the prediction image;
a prediction imaging circuitry portion that performs a filtering process to compensate the plurality of motion compensation images extracted by the extraction circuitry portion for high-frequency components by utilizing correlation in a time direction that is included in the plurality of motion compensation images, thereby generating the prediction image having a higher resolution than the plurality of motion compensation images,
wherein the prediction imaging circuitry portion includes:
a difference calculation circuitry portion that calculates a difference image between the plurality of motion compensation images extracted by the extraction circuitry portion, and
an adder that adds an image obtained by applying a low-pass filter to the difference image and another image obtained by applying a high-pass filter to the image obtained by applying the low-pass filter to one of the plurality of motion compensation images extracted by the extraction circuitry portion, and that generates the prediction image.

5. The image processing apparatus according to claim 4 ,
wherein when decoding of a high-resolution layer is to be performed, the extraction circuitry portion uses, as the reference frames, frames in a layer having a lower resolution than the high-resolution layer, and extracts the plurality of motion compensation images from the reference frames in the layer having the lower resolution using motion vectors detected in the layer having the lower resolution by the detecting circuitry portion, and
wherein the prediction imaging circuitry portion generates the prediction image in the high-resolution layer by performing the filtering process on the plurality of motion compensation images extracted from the reference frames in the layer having the lower resolution.

6. An image processing apparatus comprising:
processing circuitry including:
a detecting circuitry portion that detects motion vectors based on images and an original image that is an image to be encoded, each of the images being obtained by performing local decoding based on a residual signal indicating a difference between the original image and a prediction image;
an extraction circuitry portion that performs motion compensation using, as reference frames, frames formed of the locally decoded images and using the motion vectors detected by the detecting circuitry portion, and that extracts a plurality of motion compensation images having a lower resolution than the prediction image from the reference frames corresponding to the prediction image; and
a prediction imaging circuitry portion that performs a filtering process to compensate the plurality of motion compensation images extracted by the extraction circuitry portion for high-frequency components by utilizing correlation in a time direction that is included in the plurality of motion compensation images, thereby generating the prediction image having a higher resolution than the plurality of motion compensation images,
wherein the prediction imaging circuitry portion includes:
a converter that converts a resolution of a difference image between the plurality of motion compensation images extracted by the extraction circuitry portion and that increases the resolution of the difference image,
a first filter that applies a low-pass filter to the difference image whose resolution has been increased by the converter,
a second filter that applies a high-pass filter to an image obtained by applying the low-pass filter by the first filter, and
an adder that adds the image obtained by applying the low-pass filter by the first filter and another image obtained by applying the high-pass filter by the second filter to one of the plurality of moti

Assignees

• Sony Corp

Inventors

• Nakagami Ohji
• Yagasaki Yoichi

Classifications

• H04N19/80

  Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence) · CPC title

• H04N19/44

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04N19/13

  Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC] · CPC title

• H04N19/573

  Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction · CPC title

• H04N19/40

  using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream · CPC title