Dynamic subtitle enhancement

Patent metadata

Field	Value
Publication number	US-11295497-B2
Application number	US-201916694207-A
Country	US
Kind code	B2
Filing date	Nov 25, 2019
Priority date	Nov 25, 2019
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

A method for modifying a subtitle of a video frame. Subtitle text representing audio content of a video frame and font attributes of the subtitle text are received. A color of the subtitle text is identified, based on the font attributes of the subtitle text. A geometric region within the video frame corresponding to a display area of the subtitle text. A predominant color of the geometric region within the video frame is calculated. A level of similarity between the color of the subtitle text and the predominant color of the geometric region is determined. Rendering information of the subtitle text is calculated, based on the level of similarity, and a modification of the subtitle text is performed by applying the rendering information to the subtitle text, such that the modification reduces the similarity level during display of the video frame.

First claim

What is claimed is:

1. A method for modifying a subtitle of a video frame, the method comprising:
one or more processors receiving subtitle text representing audio content of a video frame and font attributes associated with the subtitle text;
the one or more processors identifying a color associated with the subtitle text, based on the font attributes associated with the subtitle text;
the one or more processors calculating within the video frame an occupancy area of the subtitle text;
the one or more processors determining portions of the subtitle text, wherein a portion of the subtitle text includes a character of the subtitle text;
the one or more processors determining a predominant color of a background within a portion of the occupancy area of the subtitle text, wherein the portion of the occupancy area includes a predominant color of the background bordering the character of the portion of the subtitle text;
the one or more processors determining a level of similarity between the color associated with the character of the portion of the subtitle text and the predominant color of the background bordering the character within the portion of the occupancy area; and
the one or more processors performing a modification to the character of the subtitle text corresponding to the portion of the occupancy area, based on the level of similarity determined between the predominant color of the portion of the occupancy area bordering the character and the color of the character of the portion of the subtitle text.

2. The method of claim 1 , further comprising:
the one or more processors rendering the modification of the portion of the subtitle text within the video frame, wherein the modification changes the set of characters of the subtitle text to a color in contrast to the background color bordering the portion subtitle text; and
the one or more processors transmitting the video frame rendered with the modified subtitle text to a video display device.

3. The method of claim 1 , further comprising:
the one or more processors rendering the modification of the subtitle text with the video frame, wherein the modification reduces the level of similarity during display of the video frame, and
the one or more processors transmitting the enhanced subtitle text to storage separate from the video frame.

4. The method of claim 1 , wherein the determination of the portion within the occupancy area is based on determining the level of similarity at a subtitle text granularity of a sentence, one or more words, and one or more characters.

5. The method of claim 1 , further comprising:
the one or more processors calculating modification information of the subtitle text, based on font attributes of the subtitle text within the occupancy area of the video frame, and the color associated with respective portions of a background area bordering the portion within the occupancy area.

6. The method of claim 1 , wherein the subtitle text is processed in subsets of characters tokenized from an entire set of characters of the subtitle text of the video frame.

7. The method of claim 1 , further comprising:
the one or more processors calculating a predominant color of a background area corresponding to a sentence of the subtitle text, as a sentence portion of the background;
the one or more processors calculating a predominant color of a background area corresponding to individual words of the subtitle text as word portions of the background;
the one or more processors calculating a predominant color of a background area corresponding to characters of the subtitle text as character portions of the background; and
the one or more processors determining the level of similarity based on a predominant color of the sentence, the individual words, and the characters of the subtitle text.

8. The method of claim 1 , wherein performing the modification of the subtitle text includes selecting a modification strategy based on the level of similarity.

9. The method of claim 8 , wherein performing the modification of the subtitle text includes selecting the modification strategy from a group consisting of: leaving an original color of the subtitle text unchanged, modifying a tone of the original color of the subtitle text, modifying the subtitle text with a contrasting outline font stroke, and modifying the subtitle text by changing a color of the occupancy area within the video frame forming a shape surrounding the subtitle text to a new color in contrast to the original color of the subtitle text.

10. The method of claim 1 , wherein determining the level of similarity includes calculating a distance in a color space between the color of the subtitle text and the color of the background bordering the set of characters within the portion of the occupancy area of the subtitle text.

11. The method of claim 1 , wherein determining the level of similarity includes information associated with a display device settings and lighting conditions of an environment of the display device.

12. A computer program product for modifying a subtitle of a video frame, the computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising:
program instructions to receive subtitle text representing audio content of a video frame and font attributes associated with the subtitle text;
program instructions to identify a color associated with the subtitle text, based on the font attributes associated with the subtitle text;
program instructions to calculate within the video frame an occupancy area of the subtitle text;
program instructions to determine portions of the subtitle text, wherein a portion of the subtitle text includes a character of the subtitle text;
program instructions to determine a predominant color of a background within a portion of the occupancy area of the subtitle text, wherein the portion of the occupancy area includes a predominant color of the background bordering the character of the portion of the subtitle text;
program instructions to determine a level of similarity between the color associated with the character of the portion of the subtitle text and the predominant color of the background bordering the character within the portion of the occupancy area; and
program instructions to perform a modification to the character of the subtitle text corresponding to the portion of the occupancy area, based on the level of similarity determined between the predominant color of the portion of the occupancy area bordering the character and the color of the character of the portion of the subtitle text.

13. The computer program product of claim 12 , further comprising:
program instructions to render the modification of the portion of the subtitle text within the video frame, wherein the modification changes the set of characters of the subtitle text to a color in contrast to the background color bordering the portion subtitle text; and
program instructions to transmit the video frame rendered with the modification of the subtitle text to a video display device.

14. The computer program product of claim 12 , wherein the determination of the portion within the occupancy area is based on determining the level of similarity at a subtitle text granularity of a sentence, one or more words, and one or more characters.

15. The computer program product of claim 12 , further comprising:
program instructions to calculate modification information of the subtitle text, based on font attributes of the subtitle text wit

Assignees

• IBM

Inventors

• Castellucci Antonio
• Morgantini Daniele
• Girolami Carola
• Anselmi Giorgio

Classifications

• G06T7/90

  Determination of colour characteristics · CPC title

• H04N5/278 Primary

  Subtitling · CPC title

• G06T7/33

  using feature-based methods · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title