Techniques for machine language translation of text from an image based on non-textual context information from the image

Patent metadata

Field	Value
Publication number	US-9436682-B2
Application number	US-201414313670-A
Country	US
Kind code	B2
Filing date	Jun 24, 2014
Priority date	Jun 24, 2014
Publication date	Sep 6, 2016
Grant date	Sep 6, 2016

Abstract

A computer-implemented technique can include receiving, at a server from a mobile computing device, the server having one or more processors, an image including a text. The technique can include obtaining, at the server, optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image. The technique can include identifying, at the server, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself and (ii) being indicative of a context of the image. The technique can include based on the non-textual context information, obtaining, at the server, a translation of the OCR text to a target language to obtain a translated OCR text. The technique can include outputting, from the server to the mobile computing device, the translated OCR text.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, at a server from a mobile computing device, the server having one or more processors, an image including a text;
obtaining, at the server, optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image;
identifying, at the server, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a color of an object in the image;
based on the color of the object, determining, at the server, whether the image was captured indoors or outdoors;
based on (i) the non-textual context information and (ii) whether the image was captured indoors or outdoors, obtaining, at the server, a translation of the OCR text to a target language to obtain a translated OCR text; and
outputting, from the server to the mobile computing device, the translated OCR text.

2. The computer-implemented method of claim 1 , further comprising:
obtaining, at the server, a translation of the OCR text to the target language to obtain a baseline translated OCR text; and
adjusting, at the server, the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.

3. The computer-implemented method of claim 1 , further comprising determining, at the server, a source language of the text based on the non-textual context information, wherein the translated OCR text is further based on the source language.

4. The computer-implemented method of claim 1 , further comprising determining, at the server, a type of location at which the image was captured based on the non-textual context information, wherein the translated OCR text is further based on the type of location.

5. The computer-implemented method of claim 1 , further comprising determining, at the server, a geo-location of the mobile computing device, wherein the translated OCR text is further based on the geo-location of the mobile computing device.

6. The computer-implemented method of claim 5 , further comprising:
obtaining, at the server, map information based on the geo-location; and
identifying, at the server, points of interest near the geo-location using the map information,
wherein the translated OCR text is further based on the points of interest near the geo-location.

7. The computer-implemented method of claim 1 , further comprising determining, at the server, a user history corresponding to a user of the mobile computing device, wherein the translated OCR text is further based on the user history.

8. The computer-implemented method of claim 1 , wherein the non-textual context information includes a font of the text.

9. The computer-implemented method of claim 1 , wherein the non-textual context information includes a shape of the object.

10. A server having one or more processors configured to perform operations comprising:
receiving, from a mobile computing device, an image including a text;
obtaining optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image;
identifying non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a color of an object in the image;
based on the color of the object, determining whether the image was captured indoors or outdoors;
based on (i) the non-textual context information and (ii) whether the image was captured indoors or outdoors, obtaining a translation of the OCR text to a target language to obtain a translated OCR text; and
outputting, to the mobile computing device, the translated OCR text.

11. The server of claim 10 , wherein the operations further comprise:
obtaining a translation of the OCR text to the target language to obtain a baseline translated OCR text; and
adjusting the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text.

12. The server of claim 10 , wherein the operations further comprise determining a source language of the text based on the non-textual context information, wherein the translated OCR text is further based on the source language.

13. The server of claim 10 , wherein the operations further comprise determining a type of location at which the image was captured based on the non-textual context information, wherein the translated OCR text is further based on the type of location.

14. The server of claim 10 , wherein the operations further comprise determining a geo-location of the mobile computing device, wherein the translated OCR text is further based on the geo-location of the mobile computing device.

15. The server of claim 14 , wherein the operations further comprise:
obtaining map information based on the geo-location; and
identifying points of interest near the geo-location using the map information,
wherein the translated OCR text is further based on the points of interest near the geo-location.

16. The server of claim 10 , wherein the operations further comprise determining a user history corresponding to a user of the mobile computing device, wherein the translated OCR text is further based on the user history.

17. The server of claim 10 , wherein the non-textual context information includes at least one of (i) a font of the text and (ii) a shape of the object.

Assignees

• Google Inc

Inventors

• Mauser Arne
• Cuthbert Alexander Jay
• Denero John Sturdy

Classifications

• G06V30/262

  using context analysis, e.g. lexical, syntactic or semantic context · CPC title

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06V30/10

  Character recognition · CPC title

• G06K9/00671

  Physics · mapped topic

• G06K2209/01

  Physics · mapped topic