Systems and methods for appearance mapping for compositing overlay graphics

Patent metadata

Field	Value
Publication number	US-10497162-B2
Application number	US-201816037576-A
Country	US
Kind code	B2
Filing date	Jul 17, 2018
Priority date	Feb 21, 2013
Publication date	Dec 3, 2019
Grant date	Dec 3, 2019

Abstract

Systems and methods for overlaying a second image/video data onto a first image/video data are described herein. The first image/video data may be intended to be rendered on a display with certain characteristics—e.g., HDR, EDR, VDR or UHD capabilities. The second image/video data may comprise graphics, closed captioning, text, advertisement—or any data that may be desired to be overlaid and/or composited onto the first image/video data. The second image/video data may be appearance mapped according to the image statistics and/or characteristics of the first image/video data. In addition, such appearance mapping may be made according to the characteristics of the display that the composite data is to be rendered. Such appearance mapping is desired to render a composite data that is visually pleasing to a viewer, rendered upon a desired display.

First claim

The invention claimed is:

1. A method for overlaying a second image over a plurality of first images, the method comprising:
receiving said plurality of first images and said second image, said first images differing in dynamic range and size from said second image, the plurality of first images being temporally related and forming a first scene;
mapping a dynamic range of the first scene to a dynamic range of a display to generate adjusted first images, wherein said mapped dynamic range of the first scene is different from a maximum dynamic range of the display;
performing appearance mapping of said second image to determine an adjusted second image that differs in dynamic range from said second image, wherein performing appearance mapping of said second image (i) is performed according to mapping information about the mapped dynamic range of the first scene and image statistics regarding the second image, the image statistics including at least one of average luminance, minimum luminance, mean luminance, maximum luminance, dynamic range, reflective white, color temperature, whitepoint, color gamut and color palette, and (ii) involves selecting a dynamic range based on both the dynamic range of the first scene and the capabilities of the display; and
forming a composite image by overlaying said adjusted second image onto a portion of at least one of said adjusted first images.

2. The method of claim 1 wherein said first images comprise one of a group, said group comprising: High Dynamic Range (HDR) images, Extended Dynamic Range (EDR) images and Visual Dynamic Range (VDR) images.

3. The method according to claim 1 , wherein all method steps are repeated for at least one further scene of first images, to implement a scene-by-scene overlay method.

4. The method of claim 1 wherein said second image comprises one of a group, said group comprising: graphics, close captioning, text, video overlay, Graphical User Interface (GUI), television (TV) guides, signals from broadcasting company, signals from content creation, subtitles, signals from set top box and signals from Audio-Video (AV) receiver.

5. The method of claim 1 wherein performing appearance mapping further comprises:
mapping the dynamic range of said second image in accordance with the dynamic range of said adjusted first images.

6. The method of claim 1 wherein performing appearance mapping further comprises:
mapping the color gamut of said second image in accordance with the color gamut of said first adjusted images.

7. The method of claim 1 wherein performing appearance mapping further comprises:
mapping the luminance of said second image in accordance with the luminance of said first adjusted images.

8. The method of claim 1 wherein said method further comprises:
receiving metadata regarding said display characteristics.

9. The method of claim 8 wherein said metadata regarding said display's characteristics comprises one of a group, said group comprising: average luminance, minimum luminance, mean luminance, maximum luminance, reflective white, color temperature, whitepoint and color gamut.

10. The method of claim 9 wherein performing appearance mapping further comprises:
mapping said second image in accordance with said metadata regarding said display's characteristics.

11. The method of claim 1 wherein said method further comprises:
performing dynamic mapping of a third image to composite said third image with said first images, said third image intended to replace a part of said second image previously composited with said first images.

12. A system for overlaying a second image over a plurality of first images, said system comprising:
a display management module capable of receiving said plurality of first images, which are temporally related and form a first scene;
a compositor module capable of:
receiving said second image, the second image differing in size and dynamic range from each of the plurality of the first images;
mapping a dynamic range of the first scene to a dynamic range of a display to generate adjusted first images, wherein said mapped dynamic range of the first scene is different from a maximum dynamic range of the display; and
performing appearance mapping said second image to form an adjusted second image, said adjusted second image differing in dynamic range from said second image, wherein performing appearance mapping of said second image (i) is performed according to mapping information about the mapped dynamic range of the first scene and image statistics regarding the second image, the image statistics including at least one of average luminance, minimum luminance, mean luminance, maximum luminance, dynamic range, reflective white, color temperature, whitepoint, color gamut and color palette and (ii) involves selecting a dynamic range based on both the dynamic range of the first scene and the capabilities of the display; and

a mixing module capable of mixing said adjusted second image onto at least one of said adjusted first images to form composite images, said composite images intended to be rendered upon said display.

13. The system of claim 12 wherein said compositor module is capable of performing appearance mapping wherein said appearance mapping further comprises one of a group, said group comprising: luminance mapping, dynamic range mapping and color gamut mapping.

14. The system of claim 12 wherein said system further comprises:
a dynamic mapping module capable of replacing a part of said second image that has been composited with a first image, with a third image.

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Kunkel Timo
• Xu Ning
• Chen Tao
• Lee Bongsun
• Hulyalkar Samir N

Classifications

• H04N5/2723 Primary

  Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement (data processing systems or methods specially adapted for marketing G06Q30/02) · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• H04N21/812

  involving advertisement data (advertising per se G06Q30/02) · CPC title

• H04N5/445

  for displaying additional information (H04N5/50 takes precedence) · CPC title