Text extraction from graphical user interface content

Patent metadata

Field	Value
Publication number	US-9520102-B2
Application number	US-201313872172-A
Country	US
Kind code	B2
Filing date	Apr 29, 2013
Priority date	Apr 29, 2013
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

Systems and methods for extracting text from images rendered on a display screen, the method comprising capturing a color image rendered on a display screen; and transforming the color image to binary color image, preserving text-like graphic components and filtering out non-text-like graphical components. The transforming comprises scanning one or more areas of the color image; and detecting continuous bi-tonal regions in the scanned one or more areas, wherein the continuous bi-tonal regions have large variances.

First claim

What is claimed is:

1. A method for extracting text from images rendered on a display screen, the method comprising:
capturing a color image rendered on a display screen; and
transforming the color image to binary color image, preserving textual graphic components and filtering out non-textual graphical components, said transforming comprises:
scanning one or more areas of the color image;
detecting continuous regions in the scanned one or more areas, the continuous regions having two distinguishable shades, wherein a first shade represents the foreground and a second shade represents the background, and wherein the continuous regions comprise multiple components;
representing a set having a component from the continuous regions; and
extracting the component as text component, wherein the component is recognized based on switches between runs where a single color part of the run comprises a lower number of pixels than all other single color parts of the run.

2. The method of claim 1 , wherein the binary image comprises black and white colors.

3. The method of claim 1 , wherein the binary image is a gray scaled image.

4. The method of claim 1 , wherein the detecting comprises collecting horizontal sets of pixels in the continuous regions that contain two shades.

5. The method of claim 4 , wherein the collected horizontal sets of pixels are combined into bi-level regions.

6. The method of claim 5 , wherein pixels associated with a foreground color are distinguished from pixels from a background color, in response to determining that a bi-level region contains text.

7. A system for extracting text from images rendered on a display screen, the system comprising:
a logic unit for capturing a color image rendered on a display screen; and
a logic unit for transforming the color image to binary color image, preserving textual graphic components and filtering out non-textual graphical components, wherein the transforming comprises:
scanning one or more areas of the color image;
detecting continuous regions in the scanned one or more areas, the continuous regions having two distinguishable shades, wherein a first shade represents the foreground and a second shade represents the background, and wherein the continuous regions comprise multiple components;
representing a set having a component from the one or more continuous regions; and
extracting the component as text component, wherein the component is recognized based on switches between runs where a single color part of the run comprises a lower number of pixels than all other single color parts of the run.

8. A computer program product comprising a non-transitory computer readable storage medium having a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:
capture a color image rendered on a display screen; and
transform the color image to binary color image, preserving textual graphic components and filtering out non-textual graphical components, wherein the transforming comprises:
scanning one or more areas of the color image;
detecting continuous regions in the scanned one or more areas, the continuous regions having two distinguishable shades, wherein a first shade represents the foreground and a second shade represents the background wherein the continuous regions comprise multiple components;
representing a set having a component from the one or more continuous regions; and
extracting the component as text component, wherein the component is recognized based on switches between runs where a single color part of the run comprises a lower number of pixels than all other single color parts of the run.

Assignees

• IBM

Inventors

• Geva Amir
• Marder Mattias

Classifications

• G06V30/413 Primary

  Classification of content, e.g. text, photographs or tables · CPC title

• G09G5/02 Primary

  characterised by the way in which colour is displayed {(details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607)} · CPC title

• G06V10/267

  by performing operations on regions, e.g. growing, shrinking or watersheds · CPC title

• H04N1/40062

  Discrimination between different image types, e.g. two-tone, continuous tone · CPC title

• G06K9/342

  Physics · mapped topic