Enhancement decoder for video signals with multi-level enhancement and coding format adjustment

Patent metadata

Field	Value
Publication number	US-11653034-B2
Application number	US-201917284330-A
Country	US
Kind code	B2
Filing date	Oct 9, 2019
Priority date	Oct 9, 2018
Publication date	May 16, 2023
Grant date	May 16, 2023

Abstract

An enhancement decoder for video signals, comprising an interface to receive a first video stream (1150) using a first signal element coding format from a standard decoder, an interface to receive an enhancement data stream and a de-multiplexer (200) to decompose the enhancement data stream into a first set of enhancement data, a second set of enhancement data and a range data. A first decoder video stream derived from the first video stream at a first resolution is enhanced by a first enhancer using the first set of enhancement data. A second decoder video stream derived from an output of the first enhancer is converted by an up-sampler to a second resolution. The second resolution being higher than the first resolution. A third decoder video stream derived from an output of the up-sampler at the second resolution is enhanced by a second enhancer using the second set of enhancement data. A coding format adjustment module converts one of the first to third decoder video streams from a first signal element coding format to a second signal element coding format using the range data.

First claim

The invention claimed is:

1. An enhancement decoder for video signals, said enhancement decoder comprising:
one or more processors; and
one or more computer-readable hardware storage devices that store instructions that are executable by the one or more processors to cause the enhancement decoder to:
receive a first video stream from a standard decoder, the first video stream using a first signal element coding format;
receive an enhancement data stream;
cause a de-multiplexer to decompose the enhancement data stream into:
a first set of enhancement data,
a second set of enhancement data, and
range data, such that the range data is initially included in the enhancement data stream with the first and second sets of enhancement data and such that the range data, in addition to the first and second sets of enhancement data, is subsequently decomposed from the enhancement data stream using the de-multiplexer;

enhance a first decoder video stream, which is at a first resolution, using the first set of enhancement data, the first decoder video stream being derived from the first video stream;
after enhancing the first decoder video stream to thereby produce an enhanced video stream, convert the enhanced video stream to a second resolution, the second resolution being higher than the first resolution, wherein:
said converting includes both a spatial up-sampling operation and an operation that involves an adjustment of the range data, including an adjustment to high dynamic range (HDR) value ranges;

after converting the enhanced video stream to thereby produce a converted enhanced video stream, enhance the converted enhanced video stream, which is at the second resolution, using the second set of enhancement data.

2. The enhancement decoder of claim 1 , wherein the first video stream from the standard decoder is at a first level of quality and the second signal element coding format provides for a second level of quality, the second level of quality being higher than the first level of quality.

3. The enhancement decoder of claim 1 , wherein the first signal element coding format is a standard dynamic range (SDR) format and the second signal element coding format is a high dynamic range (HDR) format, the HDR format having a higher luminance and/or wider color range than the SDR format.

4. The enhancement decoder of claim 1 , wherein:
a second de-multiplexer is positioned upstream of said de-multiplexer,
a first output of the second de-multiplexer is fed as input into the standard decoder to generate the first video stream, and
a second output of the second de-multiplexer is the enhancement data stream.

5. The enhancement decoder of claim 1 , wherein the system is further configured to perform at least one of the following operations:
convert the first video stream from the first signal element coding format to the second signal element coding format to generate the first decoder video stream;
receive the enhanced video stream and convert the enhanced video stream from the first signal element coding format to the second signal element coding format; and
receive the converted enhanced video stream and convert the converted enhanced video stream from the first signal element coding format to the second signal element coding format.

6. The enhancement decoder of claim 1 , wherein the first signal element coding format uses a first bit length and the second signal element coding format uses a second bit length, the second bit length being longer than the first bit length.

7. The enhancement decoder of claim 1 , wherein the first set of enhancement data includes residual data.

8. The enhancement decoder of claim 1 , wherein the second set of enhancement data includes residual data.

9. The enhancement decoder of claim 1 , wherein:
one or more of the first set of enhancement data and the second set of enhancement data comprises residual data representing a difference between two corresponding video signals, and
said enhancement decoder is further configured to decode residual data within a respective one of the first set of enhancement data and the second set of enhancement data and to combine said residual data with a respective input decoder video stream to output an enhanced video stream.

10. The enhancement decoder of claim 1 , wherein the standard decoder decodes a first data stream to output the first video stream, the standard decoder implementing a standard-compliant first decoding scheme that differs from a second decoding scheme implemented by the enhancement decoder.

11. The enhancement decoder of claim 1 , wherein the first video stream comprises all frames in a video segment.

12. The enhancement decoder of claim 11 , wherein all frames of the video segment are operated on by the enhancement decoder.

13. The enhancement decoder of claim 1 , wherein converting from the first signal element coding format to the second signal element coding format is non-linear.

14. The enhancement decoder of claim 1 , wherein converting from the first signal element coding format to the second signal element coding format is non-predictive.

15. A method of decoding a video signal, said method being performed by an enhancement decoder and comprising:
receiving a first video stream from a standard decoder, the first video stream using a first signal element coding format;
receiving an enhancement data stream;
using a de-multiplexer to de-mux the enhancement data stream into:
a first set of enhancement data,
a second set of enhancement data, and
range data such that the range data is initially included in the enhancement data stream with the first and second sets of enhancement data and such that the range data, in addition to the first and second sets of enhancement data, is subsequently decomposed from the enhancement data stream using the de-multiplexer;

enhancing a first decoder video stream at a first resolution using the first set of enhancement data, the first decoder video stream being derived from the first video stream;
after enhancing the first decoder video stream to thereby produce an enhanced video stream, up-sampling the enhanced video stream to a second resolution, the second resolution being higher than the first resolution, wherein the up-sampling is combined with a range adjustment operation, which involves an adjustment to the range data, including an adjustment to high dynamic range (HDR) value ranges; and
after up-sampling the enhanced video stream to thereby produce an up-sampled enhanced video stream, enhancing the up-sampled enhanced video stream, which is at the second resolution, using the second set of enhancement data.

16. The method of claim 15 , wherein the first signal element coding format is a standard dynamic range—SDR—format and the second signal element format is a high dynamic range—HDR—format, the HDR format having a higher luminance and/or wider color range than the SDR format.

17. The method of claim 15 , wherein a second de-multiplexer is positioned upstream of said de-multiplexer, wherein a first output of the second de-multiplexer is fed as input into the standard decoder to generate the first video stream, and wherein a second output of the second de-multiplexer is the enhancement data stream.

18. The method of claim 15 , wherein said up-sampling is performed temporally by adjusting a frame rate.

19. The method of claim 15 , wherein said up-sampling is performed spatially by adjusting a pixel resolution.

20. A non-transitory computer-readable storage medium co

Assignees

• V Nova Int Ltd

Inventors

• Ferrara Simone
• Meardi Guido

Classifications

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/33

  in the spatial domain · CPC title

• H04N7/0102

  involving the resampling of the incoming video signal · CPC title

• H04N19/30 Primary

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N21/234327 Primary

  by decomposing into layers, e.g. base layer and one or more enhancement layers · CPC title