Remastering lower dynamic range content for higher dynamic range displays

Patent metadata

Field	Value
Publication number	US-12423787-B1
Application number	US-202418678542-A
Country	US
Kind code	B1
Filing date	May 30, 2024
Priority date	Mar 2, 2022
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

The technology disclosed herein involves using a machine learning model (e.g., CNN) to expand lower dynamic-range image content (e.g., SDR images) into higher dynamic-range image content (e.g., HDR images). The machine learning model can take as input the lower dynamic-range image and can output multiple expansion maps that are used to make the expanded image appear more natural. The expansion maps may be used by image operators to smooth color banding and to dim overexposed regions or user interface elements in the expanded image. The expanded content (e.g., HDR image content) may then be provided to one or more devices for display or storage.

First claim

What is claimed is:

1. A system comprising:
a memory; and
a processing device, operatively coupled with the memory, the processing device to:
process, using a trained machine learning model, an input corresponding to an image having a first pixel value range to generate a first output comprising a first expansion map in a first channel and a second output comprising a second expansion map in a second channel;
expand the image to generate image data corresponding to a second pixel value range, wherein the second pixel value range is larger than the first pixel value range;
update the image data by applying a filter to reduce one or more banding artifacts in the image data based on the first expansion map; and
update the image data by modifying a luminance of a region of the expanded image based on the second expansion map.

2. The system of claim 1 , wherein the image comprises a Standard Dynamic Range (SDR) image and the image data that is output comprises a High Dynamic Range (HDR) image.

3. The system of claim 1 , wherein the filter comprises a clamped bilateral filter of variable width and the modifying the luminance comprises applying a luminance multiplier.

4. The system of claim 1 , wherein:
the first expansion map indicates, for each pixel in the image, a distance from the pixel to be considered by the filter; and
the second expansion map indicates, for each pixel in the image, a classification of the pixel, wherein one or more pixels having a first classification are included in the region of the expanded image for which luminance is modified.

5. The system of claim 4 , wherein the first classification comprises a user interface classification, and wherein the modifying the luminance comprises reducing at least one luminance value corresponding to the region.

6. The system of claim 4 , wherein the first classification comprises a high reflectance classification, and wherein the modifying the luminance comprises increasing at least one luminance value corresponding to the region.

7. The system of claim 1 , wherein the trained machine learning model comprises a Deep Neural Network (DNN) that uses the image as input and outputs the first channel and the second channel.

8. The system of claim 1 , wherein the input further comprises one or more previous outputs of the trained machine learning model.

9. The system of claim 1 , wherein to expand the image, the processing device is to dequantize an 8 bit input image to generate at least one of: 10 bit image data, 12 bit image data, 16 bit image data, or 32 bit image data.

10. The system of claim 1 , wherein the image comprises a frame of a video that comprises a sequence of frames, and the processing device is to output, to a display device, expanded image data for at least one frame of the sequence.

11. A method comprising:
processing, by a processing device and using a trained machine learning model, an input corresponding to an image having a first pixel value range to generate a first output comprising a first expansion map in a first channel and a second output comprising a second expansion map in a second channel;
expanding the image to generate image data corresponding to a second pixel value range, wherein the second pixel value range is larger than the first pixel value range;
updating the image data by applying a filter to reduce one or more banding artifacts in the image data based on the first expansion map; and
updating the image data by modifying a luminance of a region of the expanded image based on the second expansion map.

12. The method of claim 11 , wherein the image comprises a Standard Dynamic Range (SDR) image and the image data that is output comprises a High Dynamic Range (HDR) image.

13. The method of claim 11 , wherein the filter comprises a clamped bilateral filter of variable width and the modifying the luminance comprises applying a luminance multiplier.

14. The method of claim 11 , wherein:
the first expansion map indicates, for at least one pixel in the image, a distance from the pixel to be considered by the filter; and
the second expansion map indicates, for at least one pixel in the image, a classification of the pixel, wherein one or more pixels having a first classification are included in the region of the expanded image for which luminance is modified.

15. The method of claim 14 , wherein the first classification comprises a user interface classification, and wherein the modifying the luminance comprises reducing at least one luminance value corresponding to the region.

16. The method of claim 14 , wherein the first classification comprises a high reflectance classification, and wherein the modifying the luminance comprises increasing at least one luminance value corresponding to the region.

17. A non-transitory machine-readable storage medium storing instructions which, when executed, cause a processing device to perform operations comprising:
processing, using a trained machine learning model, an input corresponding to an image having a first pixel value range to generate a first output comprising a first expansion map in a first channel and a second output comprising a second expansion map in a second channel;
expanding the image to generate image data corresponding to a second pixel value range, wherein the second pixel value range is larger than the first pixel value range;
updating the image data by applying a filter to reduce one or more banding artifacts in the image data based on the first expansion map; and
updating the image data by modifying a luminance of a region of the expanded image based on the second expansion map.

18. The non-transitory machine-readable storage medium of claim 17 , wherein the image comprises a Standard Dynamic Range (SDR) image and the image data that is output comprises a High Dynamic Range (HDR) image.

19. The non-transitory machine-readable storage medium of claim 17 , wherein the filter comprises a clamped bilateral filter of variable width and the modifying the luminance comprises applying a luminance multiplier.

20. The non-transitory machine-readable storage medium of claim 17 , wherein the trained machine learning model comprises a Convolutional Neural Network (CNN) that uses the image as input and outputs the first channel and the second channel.

Assignees

• Nvidia Corp

Inventors

• Kumar Shaveen
• Patney Anjul
• Xu Eric
• Moor Anton

Classifications

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T2207/20028

  Bilateral filtering · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title