Techniques for distributed optical character recognition and distributed machine language translation

Patent metadata

Field	Value
Publication number	US-9514377-B2
Application number	US-201414264327-A
Country	US
Kind code	B2
Filing date	Apr 29, 2014
Priority date	Apr 29, 2014
Publication date	Dec 6, 2016
Grant date	Dec 6, 2016

Abstract

A technique for selectively distributing OCR and/or machine language translation tasks between a mobile computing device and server(s) includes receiving, at the mobile computing device, an image of an object comprising a text. The mobile computing device can determine a degree of optical character recognition (OCR) complexity for obtaining the text from the image. Based on this degree of OCR complexity, the mobile computing device and/or the server(s) can perform OCR to obtain an OCR text. The mobile computing device can then determine a degree of translation complexity for translating the OCR text from its source language to a target language. Based on this degree of translation complexity, the mobile computing device and/or the server(s) can perform machine language translation of the OCR text from the source language to a target language to obtain a translated OCR text. The mobile computing device can then output the translated OCR text.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
during an image capture mode, receiving, at a mobile computing device having one or more processors, an image capture request;
in response to receiving the image capture request:
capturing, at the mobile computing device, an image of an object comprising a text in a source language;
determining, at the mobile computing device, a degree of optical character recognition (OCR) complexity for performing OCR on the image to obtain the text;
transmitting, from the mobile computing device to a server, at least a portion of the image based on the degree of OCR complexity;
receiving, at the mobile computing device from the server, OCR results; and
obtaining, at the mobile computing device, an OCR text based on the OCR results;

in response to the image capture request and obtaining the OCR text, determining, at the mobile computing device, whether to translate the OCR text to a different target language; and
in response to determining to translate the OCR text to the target language:
obtaining, at the mobile computing device, a machine language translation of the OCR text from the source language to the target language to obtain a translated OCR text;
obtaining, at the mobile computing device, a modified image by modifying (i) the image to replace the text with the translated OCR text and (ii) a styling of the translated OCR text such that its styling differs from a styling of the text; and
outputting, at a display of the mobile computing device, the modified image.

2. The computer-implemented method of claim 1 , further comprising:
performing, at the mobile computing device, OCR for the entire image when the degree of OCR complexity is less than a first OCR complexity threshold, wherein the first OCR complexity threshold represents a degree of OCR complexity that the mobile computing device is appropriate for performing itself; and
transmitting, from the mobile computing device to the server, at least the portion of the image when the degree of OCR complexity is greater than the first OCR complexity threshold.

3. The computer-implemented method of claim 2 , further comprising transmitting, from the mobile computing device to the server, all of the image when the degree of OCR complexity is greater than a second OCR complexity threshold that is greater than the first OCR complexity threshold, wherein the second OCR complexity threshold represents a degree of OCR complexity that the mobile computing device is not appropriate for performing itself.

4. The computer-implemented method of claim 3 , wherein when the degree of OCR complexity is between the first and second OCR complexity thresholds, the mobile computing device performs OCR for a first portion of the image and the mobile computing device transmits a second portion of the image to the server, the first and second portions of the image collectively forming the entire image.

5. The computer-implemented method of claim 4 , wherein OCR results for the first portion of the image that are obtained by the mobile computing device are used to generate and display the modified image before the OCR results for the second portion of the image are subsequently received from the server and used to generate and display a further modified image.

6. The computer-implemented method of claim 1 , wherein determining whether to translate the OCR text from the source language to the target language includes determining whether the source language is a preferred language of a user associated with the mobile computing device.

7. The computer-implemented method of claim 1 , wherein the styling of the translated OCR text is at least one of one of (i) italics and (ii) outlined or bordered.

8. The computer-implemented method of claim 1 , wherein the modified image is displayed during the image capture mode.

9. The computer-implemented method of claim 1 , wherein the modified image is displayed during an image preview mode that is transitioned to after capturing the image during the image capture mode.

10. A computer-implemented method, comprising:
during an image capture mode, receiving, at a mobile computing device having one or more processors, an image capture request;
in response to receiving the image capture request:
capturing, at the mobile computing device, an image of an object comprising a text in a source language;
obtaining, at the mobile computing device, optical character recognition (OCR) results for the object and the text to obtain an OCR text; and
determining, at the mobile computing device, the source language of the OCR text;

in response to receiving the image capture request and determining the source language of the OCR text, determining, at the mobile computing device, whether to translate the OCR text to a different target language; and
in response to determining to translate the OCR text to the target language:
determining, at the mobile computing device, a degree of translation complexity for performing machine language translation of the OCR text from the source language to the target language;
transmitting, from the mobile computing device to a server, at least a portion of the OCR text based on the degree of translation complexity;
receiving, at the mobile computing device from the server, machine language translation results;
obtaining, at the mobile computing device, a translated OCR text based on the machine language translation results;
obtaining, at the mobile computing device, a modified image by modifying (i) the image to replace the text with the translated OCR text and (ii) a styling of the translated OCR text such that its styling differs from a styling of the text; and
outputting, at a display of the mobile computing device, the modified image.

11. The computer-implemented method of claim 10 , further comprising:
performing, at the mobile computing device, machine language translation for the entire OCR text when the degree of translation complexity is less than a first translation complexity threshold, wherein the first translation complexity threshold represents a degree of translation complexity that the mobile computing device is appropriate for performing itself; and
transmitting, from the mobile computing device to the server, at least the portion of the OCR text when the degree of translation complexity is greater than the first translation complexity threshold.

12. The computer-implemented method of claim 11 , further comprising transmitting, from the mobile computing device to the server, all of the OCR text when the degree of translation complexity is greater than a second translation complexity threshold that is greater than the first translation complexity threshold, wherein the second translation complexity threshold represents a degree of translation complexity that the mobile computing device is not appropriate for performing itself.

13. The computer-implemented method of claim 12 , wherein when the degree of translation complexity is between the first and second translation complexity thresholds, the mobile computing device performs machine language translation for a first portion of the OCR text and the mobile computing device transmits a second portion of the OCR text to the server, the first and second portions of the OCR text collectively forming the entire OCR text.

14. The computer-implemented method of claim 13 , wherein machine language translation results for the first portion of the OCR text that are obtained by the mobile computing device are used to generate and display the modified image before the machine language translation results for the second

Assignees

• Google Inc

Inventors

• Cuthbert Alexander Jay
• Xu Peng

Classifications

• G06V10/96

  Management of image or video recognition tasks · CPC title

• G06V30/142

  using hand-held instruments; Constructional details of the instruments · CPC title

• G06V30/224 Primary

  of printed characters having additional code marks or containing code marks · CPC title

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06V30/10

  Character recognition · CPC title