Adaptive transfer function for video encoding and decoding

Patent metadata

Field	Value
Publication number	US-10812801-B2
Application number	US-201514631410-A
Country	US
Kind code	B2
Filing date	Feb 25, 2015
Priority date	Feb 25, 2014
Publication date	Oct 20, 2020
Grant date	Oct 20, 2020

Abstract

A video encoding and decoding system that implements an adaptive transfer function method internally within the codec for signal representation. A focus dynamic range representing an effective dynamic range of the human visual system may be dynamically determined for each scene, sequence, frame, or region of input video. The video data may be cropped and quantized into the bit depth of the codec according to a transfer function for encoding within the codec. The transfer function may be the same as the transfer function of the input video data or may be a transfer function internal to the codec. The encoded video data may be decoded and expanded into the dynamic range of display(s). The adaptive transfer function method enables the codec to use fewer bits for the internal representation of the signal while still representing the entire dynamic range of the signal in output.

First claim

What is claimed is:

1. A system comprising:
at least one processor;
wherein the at least one processor implements an encoder module configured to process video data according to an encoding format to generate encoded video data, wherein the encoded video data is represented using code values of an internal transfer function at a bit depth of C bits within the encoder module; and
wherein the at least one processor implements an adaptive transfer function module configured to:
receive input video data using code values of a transfer function at a bit depth of N bits, wherein N is greater than C;
determine a focus dynamic range for the input video data based at least in part according to one or more characteristics of the input video data, wherein the focus dynamic range is less than a total dynamic range of the input video data such that one or more code values of the input video data are outside the focus dynamic range, and wherein the focus dynamic range is representable by the code values of the internal transfer function at the bit depth of C bits;
crop the input video data to the focus dynamic range to generate cropped N-bit video data such that the cropped N-bit video data excludes the one or more code values of the input video data outside the focus dynamic range;
map the cropped N-bit video data according to the internal transfer function of the encoder module to generate C-bit video data; and
output the C-bit video data to the encoder module for processing.

2. The system as recited in claim 1 , wherein the adaptive transfer function module is configured to perform said receive, said determine, said crop, said map, and said output for each of one or more video frames or for each of one or more sequences of two or more video frames.

3. The system as recited in claim 1 , wherein the adaptive transfer function module is configured to perform said receive, said determine, said crop, said map, and said output for two or more regions within each of one or more video frames.

4. The system as recited in claim 1 , wherein the encoder module is further configured to include format metadata in the encoded video data, the format metadata indicating one or more parameters used in said crop.

5. The system as recited in claim 4 , further comprising:
a decoder module configured to process the encoded video data generated by the encoder module to generate decoded C-bit video data and extracted format metadata; and
an inverse adaptive transfer function module configured to expand the decoded C-bit video data according to the extracted format metadata to generate D-bit video data covering full dynamic range of a target device.

6. The system as recited in claim 5 , wherein the target device is a high dynamic range (HDR) enabled device.

7. The system as recited in claim 1 , wherein the internal transfer function is the same as the transfer function used to represent the input video data.

8. The system as recited in claim 1 , wherein the internal transfer function is different than the transfer function used to represent the input video data.

9. The system as recited in claim 1 , wherein the internal transfer function is a discontinuous transfer function representation that includes one of more portions of the transfer function used to represent the input video data and that uses a different transfer function for the remainder of the transfer function representation.

10. The system as recited in claim 1 , wherein, to determine the focus dynamic range for the input video data, the adaptive transfer function module is configured to determine the focus dynamic range for the input video data at least in part according to one or more characteristics of reference video data from one or more video frames that were previously processed by the encoder module or from one or more previously processed portions of a current video frame being processed by the encoder module.

11. The system as recited in claim 1 , wherein the one or more characteristics used to determine the focus dynamic range for the input video data include one or more of brightness, motion, texture, or color characteristics of the input video data.

12. The system as recited in claim 1 , wherein the focus dynamic range represents an effective dynamic range of the human visual system for a current scene, sequence, frame, or region of a frame.

13. The system as recited in claim 1 , wherein, to process a current video frame according to an encoding format to generate encoded video data, the encoder module is configured to access reference data from one or more previously processed video frames, wherein the system further comprises a reformatting module configured to convert the reference data from the focus dynamic range used in processing the respective video frames to the focus dynamic range determined for the current frame.

14. The system as recited in claim 1 , wherein the encoding format is one of H.264/Advanced Video Coding (AVC) format or H.265 High Efficiency Video Coding (HEVC) format.

15. A method, comprising:
iteratively performing, by an encoder module:
determining a focus dynamic range for input video data according to one or more characteristics of the input video data, wherein the input video data is represented using code values of a transfer function at a bit depth of N bits, and wherein the focus dynamic range is less than a total dynamic range of the input video data such that one or more code values of the input video data are outside the focus dynamic range, and wherein the focus dynamic range is representable by code values of a transfer function of the encoder module at a bit depth of C bits;
cropping the input video data to the focus dynamic range to generate cropped N-bit video data such that the cropped N-bit video data excludes the one or more code values of the input video data outside the focus dynamic range;
mapping the code values of the cropped N-bit video data to the code values of the transfer function of the encoder module at the bit depth of C bits to generate C-bit video data, wherein N is greater than C; and
processing the C-bit video data according to an encoding format to generate encoded video data as output.

16. The method as recited in claim 15 , further comprising:
iteratively performing, by a decoder module:
decoding input encoded video data to generate decoded C-bit video data; and
expanding the decoded C-bit video data according to format metadata to generate D-bit video data covering full dynamic range of a high dynamic range (HDR)-enabled device, wherein the format metadata indicates one or more parameters used in said cropping and said mapping.

17. The method as recited in claim 15 , wherein the focus dynamic range represents an effective dynamic range of the human visual system for a current scene, sequence, frame, or region of a frame.

18. An apparatus, comprising:
an encoder module configured to:
determine a focus dynamic range for input video data according to one or more characteristics of the input video data, wherein the input video data is represented using code values of a transfer function at a bit depth of N bits, and wherein the focus dynamic range is less than a total dynamic range of the input video data such that one or more code values of the input video data are outside the focus dynamic range, and wherein the focus dynamic range is representable by code values of a transfer function of the encoder module at a bit depth of C bits;
crop the input video data to the focus dynamic range to generate cropped N-bit video data such that the cropp

Assignees

• Apple Inc

Inventors

• Tourapis Alexandros
• Singer David

Classifications

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N9/67

  for matrixing (camera processing pipelines for matrixing of colour signals H04N23/85) · CPC title

• H04N23/85

  for matrixing · CPC title

• H04N19/98 Primary

  Adaptive-dynamic-range coding [ADRC] · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title