Font attributes for font recognition and similarity

Patent metadata

Field	Value
Publication number	US-9875429-B2
Application number	US-201514876667-A
Country	US
Kind code	B2
Filing date	Oct 6, 2015
Priority date	Oct 6, 2015
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

Font recognition and similarity determination techniques and systems are described. In a first example, localization techniques are described to train a model using machine learning (e.g., a convolutional neural network) using training images. The model is then used to localize text in a subsequently received image, and may do so automatically and without user intervention, e.g., without specifying any of the edges of a bounding box. In a second example, a deep neural network is directly learned as an embedding function of a model that is usable to determine font similarity. In a third example, techniques are described that leverage attributes described in metadata associated with fonts as part of font recognition and similarity determinations.

First claim

What is claimed is:

1. In a digital medium environment to recognize a font in rendered text in an image or determine similarity of the font in the rendered text in the image to other fonts, a method implemented by one or more computing devices comprising:
obtaining a model, by the one or more computing devices, that is trained using machine learning based at least in part on training data that includes one or more relative attributes that define a relationship of at least four fonts within a font family to each other, the at least four fonts in the font family comprising an italic font and a bold font, the one or more relative attributes including a font weight attribute for the font family learned by comparing the fonts in the font family, one to another;
using the model, by the one or more computing devices, to:
predict a weight of the font in the rendered text in the image using a weight prediction function that includes the font weight attribute; and
recognize the font used for the rendered text in the image; or
determine similarity of the font used for the rendered text in the image with respect to one or more of a plurality of fonts by comparing the predicted weight of the font used for the rendered text in the image to another predicted weight of at least one font in a different font family; and

controlling output of a result of the recognized font or the determined similarity in a user interface by the one or more computing devices.

2. The method as described in claim 1 , wherein the model is also learned using categorical attributes that describe relationships of font qualities between font families.

3. The method as described in claim 1 , wherein the one or more relative attributes include a weight that describes a thickness of a line used to form a respective said font.

4. The method as described in claim 1 , wherein the one or more relative attributes include regular/italic pairs.

5. The method as described in claim 1 , wherein the machine learning is performed using a Siamese neural network structure.

6. The method as described in claim 5 , wherein the Siamese neural network structure is configured to incorporate categorical and relative attributes for both font recognition and font similarity determination.

7. The method as described in claim 5 , wherein the model is further trained using ordered pairs of images that include text rendered using fonts in the font family, and wherein a first image is of a first font having a smaller weight and a second font having a larger weight and forming a positive sample, and wherein a second image is of a third font having a larger weight and a fourth font having a smaller weight and forming a negative sample.

8. The method as described in claim 7 , wherein the Siamese network generates scalar weight predictions for each of the first image and the second image and modulates a margin parameter between the scalar weight predictions based on the font weight attribute.

9. The method as described in claim 8 , further comprising normalizing the scalar weight predictions after a last linear layer of the Siamese network.

10. In a digital medium environment to generate a model usable to recognize a font in rendered text in an image or determine similarity of the font in the image to other fonts, a method implemented by one or more computing devices comprising:
extracting attributes from font metadata by the one or more computing devices;
controlling training of the model using machine learning by the one or more computing devices based at least in part on the extracted attributes and one or more relative attributes that define a relationship of at least four fonts within a font family to each other, the at least four fonts in the font family comprising an italic font and a bold font, the one or more relative attributes including a font weight attribute for the font family learned by comparing the fonts in the font family, one to another, the trained model configured to:
predict a weight of the font in the rendered text in the image using a weight prediction function that includes the font weight attribute; and
recognize the font used for the rendered text in the image; or
determine similarity of the font used for the rendered text in the image with respect to one or more of a plurality of fonts by comparing the predicted weight of the font used for the rendered text in the image to another predicted weight of at least one font in a different font family.

11. The method as described in claim 10 , wherein the one or more relative attributes include a weight that describes a thickness of a line used to form a respective said font.

12. The method as described in claim 10 , wherein the one or more relative attributes include regular/italic pairs.

13. The method as described in claim 10 , wherein the extracted attributes are categorical attributes that describe relationships of font qualities between font families.

14. The method as described in claim 10 , wherein the machine learning is performed using a Siamese neural network structure.

15. The method as described in claim 14 , wherein the Siamese neural network structure is configured to incorporate categorical and relative attributes for both font recognition and font similarity determination.

16. In a digital medium environment to recognize a font in rendered text in an image or determine similarity of the font in the rendered text in the image to other fonts, a system comprising one or more computing devices including a processing system and memory having instructions stored thereon that are executable by the processing system to perform operations comprising:
obtaining a model that is trained using machine learning based at least in part on training data that includes one or more attributes extracted from font metadata and one or more relative attributes that define a relationship of at least four fonts in a font family to each other, the at least four fonts in the font family comprising an italic font and bold font, the one or more relative attributes including a font weight attribute for the font family learned by comparing the fonts in the font family, one to another;
using the model to:
predict a weight of the font in the rendered text in the image using a weight prediction function that includes the font weight attribute; and
recognize the font used for the rendered text in the image; or
determine similarity of the font used for the rendered text in the image with respect to one or more of a plurality of fonts by comparing the predicted weight of the font used for the rendered text in the image to another predicted weight of at least one font in a different font family; and

controlling output of a result of the recognized font or the determined similarity in a user interface by the one or more computing devices.

17. The system as described in claim 16 , wherein the one or more relative attributes include a weight that describes a thickness of a line used to form a respective said font.

18. The system as described in claim 16 , wherein the one or more relative attributes include regular/italic pairs.

19. The system as described in claim 16 , wherein the extracted attributes are categorical attributes that describe relationships of font qualities between font families.

20. The system as described in claim 16 , wherein the machine learning is performed using a Siamese neural network structure configured to incorporate categorical and relative attributes for both font recognition and font similarity determinatio

Assignees

• Adobe Systems Inc

Inventors

• Wang Zhaowen
• Liu Luoqi
• Jin Hailin

Classifications

• G06V30/18057

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F18/24137

  Distances to cluster centroïds · CPC title