Two-layered image compression for text content

Patent metadata

Field	Value
Publication number	US-12542918-B2
Application number	US-202318522830-A
Country	US
Kind code	B2
Filing date	Nov 29, 2023
Priority date	Nov 29, 2023
Publication date	Feb 3, 2026
Grant date	Feb 3, 2026

Abstract

Coding an image that includes text content and a background is disclosed. Text portions are identified in the image. The text portions are extracted from the image to obtain a background image, where the background image includes holes corresponding to respective areas of the text portions within the image. A filled-in background image is obtained based on the background image. The filled-in background image is encoded into a compressed bitstream using a block-based encoder. The text portions is also encoded into the compressed bitstream. Encoding the text portions includes encoding respective high quality text binarization upscaled binary maps.

First claim

What is claimed is:

1 . A method for coding an image that includes text content and a background, comprising:
identifying text portions in the image; extracting the text portions from the image to obtain a background image, wherein the background image includes holes corresponding to respective areas of the text portions within the image; obtaining a filled-in background image based on the background image; encoding, in a compressed bitstream, a flag indicating whether the image is encoded using two-layer encoding that separately encodes the text content from the background; encoding, into the compressed bitstream, the filled-in background image using a block-based encoder; and encoding, into the compressed bitstream, the text portions.

2 . The method of claim 1 , wherein obtaining the filled-in background image based on the background image comprises:
for a text portion of the text portions:
initiating a background refill process from pixels connected to the background image;
determining a value of a current pixel by calculating an average of respective pixel values of available neighboring background pixels; and
repeatedly applying the background refill process, growing inward from pixels connected to the background image, until all pixels within the text portion are assigned respective values.

3 . The method of claim 1 , wherein encoding, into the compressed bitstream, the text portions comprises:
for a text portion of the text portions:
obtaining an anti-aliasing map based on classifying each pixel of the text portion as an edge pixel, a background pixel, or a text pixel, wherein respective values are associated with cells of the anti-aliasing map based on the classifying;
obtaining an upscaled binary map based on the anti-aliasing map, wherein a first number of sub-cells corresponding to a cell of the anti-aliasing map is indicated as being text sub-cells based on a value associated with the cell;
identifying a smooth contour for a text symbol in the text portion based on the anti-aliasing map;
modifying the upscaled binary map to obtain a modified upscaled binary map based on the smooth contour by setting sub-cells within the text symbol and crossed by the smooth contour to indicate text sub-cells and setting other sub-cells to indicate background sub-cells; and
encoding the modified upscaled binary map in the compressed bitstream.

4 . The method of claim 3 , wherein classifying the each pixel is based on an average background pixel value within the text portion and a maximum pixel value within the text portion.

5 . The method of claim 4 , wherein the average background pixel value is calculated based on pixels along a perimeter of the text portion.

6 . The method of claim 1 , wherein encoding, into the compressed bitstream, the text portions comprises:
identifying connected symbols within a text portion by applying a set of kernels to detect patterns of possible connections between adjacent symbols.

7 . The method of claim 1 , wherein the text portions in the image are identified using optical character recognition (OCR).

8 . The method of claim 1 , wherein encoding, into the compressed bitstream, the text portions comprises:
encoding in the compressed bitstream:
a color palette;
a dictionary of symbols;
indexes of the symbols in the dictionary of symbols; and
color indexes into the color palette.

9 . A device for coding an image that includes text content and a background, comprising:
a processor configured to:
identify text portions in the image;
extract the text portions from the image to obtain a background image, wherein the background image includes holes corresponding to respective areas of the text portions within the image;
obtain a filled-in background image based on the background image;
encode, in a compressed bitstream, a flag indicating whether the image is encoded using two-layer encoding that separately encodes the text content from the background;
encode, into the compressed bitstream, the filled-in background image using a block-based encoder; and
encode, into the compressed bitstream, the text portions.

10 . The device of claim 9 , wherein to obtain the filled-in background image based on the background image comprises to:
for a text portion of the text portions:
initiate a background refill process from pixels connected to the background image;
determine a value of a current pixel by calculating an average of respective pixel values of available neighboring background pixels; and
repeatedly apply the background refill process, growing inward from pixels connected to the background image, until all pixels within the text portion are assigned respective values.

11 . The device of claim 9 , wherein to encode, into the compressed bitstream, the text portions comprises to:
for a text portion of the text portions:
obtain an anti-aliasing map based on classifying each pixel of the text portion as an edge pixel, a background pixel, or a text pixel, wherein respective values are associated with cells of the anti-aliasing map based on the classifying;
obtain an upscaled binary map based on the anti-aliasing map, wherein a first number of sub-cells corresponding to a cell of the anti-aliasing map is indicated as being text sub-cells based on a value associated with the cell;
identify a smooth contour for a text symbol in the text portion based on the anti-aliasing map;
modify the upscaled binary map to obtain a modified upscaled binary map based on the smooth contour by setting sub-cells within the text symbol and crossed by the smooth contour to indicate text sub-cells and setting other sub-cells to indicate background sub-cells; and
encode the modified upscaled binary map in the compressed bitstream.

12 . The device of claim 11 , wherein the each pixel is classified based on an average background pixel value within the text portion and a maximum pixel value within the text portion.

13 . The device of claim 12 , wherein the average background pixel value is calculated based on pixels along a perimeter of the text portion.

14 . The device of claim 9 , wherein to encode, into the compressed bitstream, the text portions comprises to:
identify connected symbols within a text portion by applying a set of kernels to detect patterns of possible connections between adjacent symbols.

15 . The device of claim 9 , wherein the text portions in the image are identified using optical character recognition (OCR).

16 . The device of claim 9 , wherein to encode, into the compressed bitstream, the text portions comprises to:
encode in the compressed bitstream:
a color palette;
a dictionary of symbols;
indexes of the symbols in the dictionary of symbols; and
color indexes into the color palette.

17 . A method for decoding an image comprising:
decoding, from a compressed bitstream, a flag indicating that the image is encoded using two-layer encoding that separately encodes text content of the image from a background image of the image; decoding, from the compressed bitstream, the background image of the image using a block-based decoder; decoding, from the compressed bitstream, respective locations of text symbols; and overlaying, based on the respective locations, the text symbols on the background image to obtain the image.

18 . The method of claim 17 , further comprising:
decoding, from the compressed bitstream, an upscaled binary image that includes, for each pixel of a text portion, a respective first number of a first binary value a

Assignees

• Google Llc

Inventors

• Chen Di
• Adsumilli Balineedu
• Mukherjee Debargha
• Lin Yao-Chung

Classifications

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title

• H04N19/46 Primary

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N19/186

  the unit being a colour or a chrominance component · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/119

  Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks · CPC title