Content-aware PQ range analyzer and tone mapping in live feeds

Patent metadata

Field	Value
Publication number	US-12050830-B2
Application number	US-202017606433-A
Country	US
Kind code	B2
Filing date	Apr 20, 2020
Priority date	Apr 25, 2019
Publication date	Jul 30, 2024
Grant date	Jul 30, 2024

Abstract

In image processing system comprises an input configured to receive an image signal, the image signal including a plurality of frames of image data; and a processor configured to automatically determine an image classification based on at least one frame of the plurality of frames, and dynamically generate a mapping metadata based on the image classification. The processor includes determination circuitry configured to determine a content type for the image signal; segmentation circuitry configured to segment the image data into a plurality of feature item regions, based on the content type; extraction circuitry configured to extract at least one image aspect value for respective ones of the plurality of feature item regions.

First claim

The invention claimed is:

1. An image processing system, comprising:
an input configured to receive an image signal, the image signal including a plurality of frames of image data; and
a processor configured to automatically determine an image classification based on at least one frame of the plurality of frames, and dynamically generate a mapping metadata based on the image classification,
wherein the processor includes:
determination circuitry configured to determine a content type for the image signal;
segmentation circuitry configured to segment the image data into a plurality of feature item regions, based on the content type; and
extraction circuitry configured to extract at least one image luminance value for respective ones of the plurality of feature item regions,

wherein the determination circuitry is configured to determine the content type by analyzing regions of the frame and determining one or more confidence regions;
wherein the determination of the content type involves generating a ranked or unranked list of potential content types based on the one or more confidence regions;
wherein the segmentation of the image data involves, based on the determined content type, determining an ordered set of priority items in the image data to search and segment for;
wherein the mapping metadata is dynamically generated based on the content type, the feature item regions, and/or image luminance value; and
wherein the mapping metadata includes tone and/or gamut mapping data for converting from a first dynamic range to a second dynamic range that is different than the first dynamic range.

2. The image processing system according to claim 1 , wherein the at least one image luminance value includes at least one selected from a luminance maximum, a luminance minimum, a luminance midpoint, a luminance mean, or a luminance variance.

3. The image processing system according to claim 1 , wherein a respective feature item region indicates at least one selected from a landscape region, a shadow region, a sky region, a facial detection region, or a crowd region.

4. The image processing system according to claim 1 , wherein the image signal is a live video feed.

5. The image processing system according to claim 1 , further comprising an encoder configured to encode the image signal and the mapping metadata.

6. The image processing system according to claim 1 , wherein the first dynamic range is higher than the second dynamic range.

7. An image processing method comprising:
receiving an image signal, the image signal including a plurality of frames of image data;
automatically determining an image classification based on at least one frame of the plurality of frames, including:
determining a content type for the image signal,
segmenting the image data into a plurality of feature item regions, based on the content type, and
extracting at least one image luminance value for respective ones of the plurality of feature item regions; and

generating a plurality of frames of mapping metadata based on the image classification, wherein respective ones of the plurality of frames of mapping metadata correspond to respective ones of the plurality of frames of image data,
wherein the content type is determined by analyzing regions of the frame and determining one or more confidence regions;
wherein the determination of the content type involves generating a ranked or unranked list of potential content types based on the one or more confidence regions;
wherein the segmentation of the image data involves, based on the determined content type, determining an ordered set of priority items in the image data to search and segment for;
wherein the mapping metadata is dynamically generated based on the content type, the feature item regions, and/or image luminance value; and
wherein the mapping metadata includes tone and/or gamut mapping data for converting from a first dynamic range to a second dynamic range that is different than the first dynamic range.

8. The image processing method according to claim 7 , wherein the at least one image luminance value includes at least one selected from a luminance maximum, a luminance minimum, a luminance midpoint, a luminance mean, or a luminance variance.

9. The image processing method according to claim 7 , wherein a respective feature item region indicates at least one selected from a landscape region, a shadow region, a sky region, a facial detection region, or a crowd region.

10. The image processing method according to claim 7 , wherein the image signal is a live video feed.

11. The image processing method according to claim 7 , further comprising encoding the image signal and the mapping metadata into a compressed output signal.

12. The image processing method according to claim 7 , wherein the first dynamic range is higher than the second dynamic range.

13. A non-transitory computer-readable medium storing instructions that, when executed by a processor of an image processing system, cause the image processing system to perform operations comprising:
receiving an image signal, the image signal including a plurality of frames of image data;
automatically determining an image classification based on at least one frame of the plurality of frames, including:
determining a content type for the image signal,
segmenting the image data into a plurality of feature item regions, based on the content type, and
extracting at least one image luminance value for respective ones of the plurality of feature item regions; and

dynamically generating a mapping metadata based on the image classification on a frame-by-frame basis,
wherein the content type is determined by analyzing regions of the frame and determining one or more confidence regions;
wherein the determination of the content type involves generating a ranked or unranked list of potential content types based on the one or more confidence regions;
wherein the segmentation of the image data involves, based on the determined content type, determining an ordered set of priority items in the image data to search and segment for;
wherein the mapping metadata is dynamically generated based on the content type, the feature item regions, and/or image luminance value; and
wherein the mapping metadata includes tone and/or gamut mapping data for converting from a first dynamic range to a second dynamic range that is different than the first dynamic range.

14. The non-transitory computer-readable medium according to claim 13 , wherein the at least one image luminance value includes at least one selected from a luminance maximum, a luminance minimum, a luminance midpoint, a luminance mean, or a luminance variance.

15. The non-transitory computer-readable medium according to claim 13 , wherein a respective feature item region indicates at least one selected from a landscape region, a shadow region, a sky region, a facial detection region, or a crowd region.

16. The non-transitory computer-readable medium according to claim 13 , wherein the image signal is a live video feed.

17. The non-transitory computer-readable medium according to claim 13 , further comprising encoding the image signal and the mapping metadata.

18. The non-transitory computer-readable medium according to claim 13 , wherein the mapping metadata includes conversion data for converting from an HDR signal to an SDR signal.

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Zandifar Ali
• Crenshaw James E
• Vasco Cristina Michel

Classifications

• G06T5/92

  based on global image properties · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• H04N7/01

  Conversion of standards {, e.g. involving analogue television standards or digital television standards processed at pixel level} · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title