Techniques for machine language translation of text from an image based on non-textual context information from the image

Patent metadata

Field	Value
Publication number	US-2016371256-A1
Application number	US-201615252309-A
Country	US
Kind code	A1
Filing date	Aug 31, 2016
Priority date	Jun 24, 2014
Publication date	Dec 22, 2016
Grant date	—

Abstract

A computer-implemented technique can include receiving, at a server from a mobile computing device, the server having one or more processors, an image including a text. The technique can include obtaining, at the server, optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image. The technique can include identifying, at the server, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself and (ii) being indicative of a context of the image. The technique can include based on the non-textual context information, obtaining, at the server, a translation of the OCR text to a target language to obtain a translated OCR text. The technique can include outputting, from the server to the mobile computing device, the translated OCR text.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
receiving, by a server computing system and from a client computing device, an image including a text; obtaining, by the server computing system, an optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image; identifying, by the server computing system, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a font of the text; obtaining, by the server computing system, a translated OCR text based on the non-textual context information, the translated OCR text representing a translation of the OCR text to a different target language; and outputting, from the server computing system and to the client computing device, the translated OCR text.

2 . The computer-implemented method of claim 1 , further comprising determining, by the server computing system, a type of an object that is associated with the text based on the font of the text.

3 . The computer-implemented method of claim 2 , wherein obtaining the translated OCR text is based on the type of the object.

4 . The computer-implemented method of claim 2 , wherein determining the type of the object is further based on a shape of the object.

5 . The computer-implemented method of claim 1 , wherein obtaining the translated OCR text includes:
obtaining, by the server computing system, a baseline translated OCR text representing a machine translation of the OCR text to the target language; and adjusting, by the server computing system, the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.

6 . The computer-implemented method of claim 1 , further comprising determining, by the server computing system, a source language of the text based on the non-textual context information, wherein obtaining the translated OCR text is further based on the source language.

7 . The computer-implemented method of claim 1 , further comprising determining, by the server computing system, a type of location at which the image was captured based on the non-textual context information, wherein obtaining the translated OCR text is further based on the type of location.

8 . A server computing system, comprising:
a non-transitory computer-readable medium having a set of instructions stored thereon; and one or more processors configured to execute the set of instructions, which causes the server computing system to perform operations comprising:
receiving, from a client computing device, an image including a text;
obtaining optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image;
identifying non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a font of the text;
obtaining a translated OCR text based on the non-textual context information, the translated OCR text representing a translation of the OCR text to a target language; and
outputting, to the client computing device, the translated OCR text.

9 . The server computing system of claim 8 , wherein the operations further comprise determining a type of an object that is associated with the text based on the font of the text.

10 . The server computing system of claim 9 , wherein obtaining the translated OCR text is based on the type of the object.

11 . The server computing system of claim 9 , wherein determining the type of the object is further based on a shape of the object.

12 . The server computing system of claim 8 , wherein obtaining the translated OCR text includes:
obtaining a baseline translated OCR text representing a machine translation of the OCR text to the target language; and adjusting the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.

13 . The server computing system of claim 8 , wherein the operations further comprise determining a source language of the text based on the non-textual context information, and wherein obtaining the translated OCR text is further based on the source language.

14 . The server computing system of claim 8 , wherein the operations further comprise determining a type of location at which the image was captured based on the non-textual context information, and wherein obtaining the translated OCR text is further based on the type of location.

15 . A non-transitory computer-readable medium having a set of instructions stored thereon that, when executed by one or more processors of a server computing system, causes the server computing system to perform operations comprising:
receiving, from a client computing device, an image including a text; obtaining optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image; identifying non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a font of the text; obtaining a translated OCR text based on the non-textual context information, the translated OCR text representing a translation of the OCR text to a target; and outputting, to the client computing device, the translated OCR text.

16 . The computer-readable medium of claim 15 , wherein the operations further comprise determining a type of an object that is associated with the text based on the font of the text.

17 . The computer-readable medium of claim 16 , wherein obtaining the translated OCR text is based on the type of the object.

18 . The computer-readable medium of claim 16 , wherein determining the type of the object is further based on a shape of the object.

19 . The computer-readable medium of claim 15 , wherein obtaining the translated OCR text includes:
obtaining a baseline translated OCR text representing a machine translation of the OCR text to the target language; and adjusting the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.

20 . The computer-readable medium of claim 15 , wherein the operations further comprise determining a type of location at which the image was captured based on the non-textual context information, and wherein obtaining the translated OCR text is further based on the type of location.

Assignees

• Google Inc

Inventors

• Mauser Arne
• Cuthbert Alexander Jay
• Denero John Sturdy

Classifications

• G06V30/262

  using context analysis, e.g. lexical, syntactic or semantic context · CPC title

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06V30/10

  Character recognition · CPC title

• G06K2209/01

  Physics · mapped topic

• G06F17/289 Primary

  Physics · mapped topic