Apparatus and method for detecting scene text in an image

Patent metadata

Field	Value
Publication number	US-10679085-B2
Application number	US-201816170285-A
Country	US
Kind code	B2
Filing date	Oct 25, 2018
Priority date	Oct 31, 2017
Publication date	Jun 9, 2020
Grant date	Jun 9, 2020

Abstract

Computer program products, methods, systems, apparatus, and computing entities provide a unique single-shot text detector that generates word-level text bounding boxes in an image by at least identifying text regions in the image via an automatically learned attention map and by conducting pixel-wise review of text; aggregating multi-scale inception features; generating, based at least in part on the multi-scale inception features, a set of aggregated inception features; and generating, using at least the set of aggregated inception features, the word-level text bounding boxes in the image.

First claim

The invention claimed is:

1. A method comprising:
identifying, by one or more processors, text regions in an image via an automatically learned attention map and by conducting pixel-wise review of text;
aggregating, by the one or more processors, multi-scale inception features;
generating, by the one or more processors and based at least in part on the multi-scale inception features, a set of aggregated inception features; and
generating, by the one or more processors and using at least the set of aggregated inception features, word-level text bounding boxes in the image.

2. The method of claim 1 , wherein conducting the pixel-wise review of text comprises:
convoluting, by the one or more processors, the image as a whole with a convolutional neural network to generate intermediate features;
generating, by the one or more processors, the attention map via a softmax function that uses the intermediate features; and
performing, by the one or more processors, pixel-wise multiplication between the attention map and the convolutional features to suppress background interference.

3. The method of claim 2 , wherein performing the pixel-wise multiplication comprises masking the background either by features or raw images.

4. The method of claim 1 , further comprising:
aggregating, by the one or more processors, convolutional features;
capturing, by the one or more processors, context information by using multi-scale receptive fields; and
aggregating, by the one or more processors, multi-layer inception modules and enhancing the convolutional features towards text task.

5. The method of claim 1 , wherein (a) identifying the text regions is performed by a text attention module, and (b) aggregating multi-scale inception features is performed by a hierarchical inception module.

6. The method of claim 5 , wherein (a) generating the set of aggregated inception features is performed by the hierarchical inception module and (b) generating the word-level text bounding boxes in the image is performed by a word prediction module.

7. A apparatus comprising at least a processor, and a memory associated with the processor having computer coded instructions therein, with the computer instructions configured to, when executed by the processor, cause the apparatus to:
identify text regions in an image via an automatically learned attention map and by conducting pixel-wise review of text;
aggregate multi-scale inception features;
generate, based at least in part on the multi-scale inception features, a set of aggregated inception features; and
generate, using at least the set of aggregated inception features, word-level text bounding boxes in the image.

8. The apparatus of claim 7 , wherein the conducting the pixel-wise review of text comprises:
convolute the image as a whole with a convolutional neural network to generate intermediate features;
generate the attention map via a softmax function that uses the intermediate features; and
perform pixel-wise multiplication between the attention map and the convolutional features to suppress background interference.

9. The apparatus of claim 8 , wherein performing the pixel-wise multiplication comprises masking the background either by features or raw images.

10. The apparatus of claim 7 , further comprising:
aggregate convolutional features;
capture context information by using multi-scale receptive fields; and
aggregate multi-layer inception modules and enhancing the convolutional features towards text task.

11. The apparatus of claim 7 , wherein (a) identifying the text regions is performed by a text attention module, and (b) aggregating multi-scale inception features is performed by a hierarchical inception module.

12. The apparatus of claim 7 , wherein (a) generating the set of aggregated inception features is performed by the hierarchical inception module and (b) generating the word-level text bounding boxes in the image is performed by a word prediction module.

13. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:
an executable portion configured to identify text regions in an image via an automatically learned attention map and by conducting pixel-wise review of text;
an executable portion configured to aggregate multi-scale inception features;
an executable portion configured to generate, based at least in part on the multi-scale inception features, a set of aggregated inception features; and
an executable portion configured to generate, using at least the set of aggregated inception features, word-level text bounding boxes in the image.

14. The computer program product of claim 13 , wherein the conducting the pixel-wise review of text comprises:
convolute the image as a whole with a convolutional neural network to generate intermediate features;
generate the attention map via a softmax function that uses the intermediate features; and
perform pixel-wise multiplication between the attention map and the convolutional features to suppress background interference.

15. The computer program product of claim 14 , wherein performing the pixel-wise multiplication comprises masking the background either by features or raw images.

16. The computer program product of claim 13 , further comprising:
an executable portion aggregate convolutional features;
an executable portion capture context information by using multi-scale receptive fields; and
an executable portion aggregate multi-layer inception modules and enhancing the convolutional features towards text task.

17. The computer program product of claim 13 , wherein (a) identifying the text regions is performed by a text attention module, and (b) aggregating multi-scale inception features is performed by a hierarchical inception module.

18. The computer program product of claim 13 , wherein (a) generating the set of aggregated inception features is performed by the hierarchical inception module and (b) generating the word-level text bounding boxes in the image is performed by a word prediction module.

Assignees

• The Univ Of Florida Research Foundation Incorporated
• Univ Florida

Inventors

• Li Xiaolin
• He Pan

Classifications

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06K2209/01

  Physics · mapped topic

• G06K9/6274

  Physics · mapped topic

• G06K9/4671

  Physics · mapped topic

• G06K9/4628

  Physics · mapped topic