Techniques for providing user image capture feedback for improved machine language translation

Patent metadata

Field	Value
Publication number	US-2016203124-A1
Application number	US-201514594238-A
Country	US
Kind code	A1
Filing date	Jan 12, 2015
Priority date	Jan 12, 2015
Publication date	Jul 14, 2016
Grant date	—

Abstract

A computer-implemented technique includes techniques are presented for user image capture feedback for improved machine language translation. When machine language translation of OCR text obtained from an initial image has a low degree of likelihood of being an appropriate translation, these techniques provide for user image capture feedback to obtain additional images to obtain a modified OCR text, which can result in improved machine language translation results. Instead of user image capture feedback, the techniques may obtain the modified OCR text by selecting another possible OCR text from the initial OCR operation. In addition to additional image capturing, light source intensity and/or a quantity/number of light source flashes can be adjusted. After obtaining the modified OCR text, another machine language translation can be obtained and, if it has a high enough degree of likelihood, it can then be output to a user.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
receiving, at a computing system having one or more processors, a first image of an object comprising a text in a source language, the first image being captured by a camera in communication with the computing system; performing, by the computing system, optical character recognition (OCR) on the first image to obtain an OCR text that is a machine-encoded text representation of the text; obtaining, by the computing system, a first translated OCR text and a translation score indicative of a degree of likelihood that the first translated OCR text is an appropriate translation of the OCR text to a target language; and when the translation score is less than a translation score threshold indicative of an acceptable degree of likelihood:
outputting, by the computing system, a user instruction to capture a set of second images of at least a portion of the object using the camera;
receiving, at the computing system, the set of second images;
performing, by the computing system, OCR on at least one of the set of second images to obtain a modified OCR text corresponding to the text;
in response to obtaining the modified OCR text, obtaining, by the computing system, a second translated OCR text representing a translation of the modified OCR text from the source language to the target language; and
outputting, by the computing system, the second translated OCR text.

2 . The computer-implemented method of claim 1 , further comprising capturing, by the computing system, the first image and the set of second images using the camera.

3 . The computer-implemented method of claim 2 , further comprising:
when the translation score is less than the translation score threshold, adjusting, by the computing system, an intensity of a light source associated with the camera while capturing the second set of images, the light source being in communication with the computing system.

4 . The computer-implemented method of claim 2 , further comprising:
when the translation score is less than the translation score threshold, adjusting, by the computing system, a number of flashes of a light source associated with the camera while capturing the second set of images, the light source being in communication with the computing system.

5 . The computer-implemented method of claim 1 , further comprising:
when the translation score is less than the translation score threshold, identifying, by the computing system, a portion of the first image causing the OCR text and the corresponding first translated OCR text to have the translation score less than the translation score threshold, wherein the user instruction is to capture the set of second images with respect to the identified portion of the first image.

6 . The computer-implemented method of claim 5 , wherein performing OCR on at least one of the set of second images to obtain a modified OCR text corresponding to the text comprises:
combining portions of at least two of the first image and the set of second images to obtain a combined image; and performing OCR on the combined image to obtain the modified OCR text.

7 . The computer-implemented method of claim 1 , wherein performing OCR on the first image to obtain the OCR text includes:
performing, by the computing system, OCR on the first image to obtain a set of possible OCR texts, wherein each possible OCR text has a corresponding OCR score indicative of a degree of likelihood that the possible OCR text is the text; and selecting, by the computing system, the possible OCR text having a highest corresponding OCR score relative to the OCR scores of the other possible OCR texts to obtain the OCR text.

8 . The computer-implemented method of claim 7 , further comprising when the translation score is less than the translation score threshold, selecting, by the computing system, another one of the possible OCR texts to obtain the modified OCR text.

9 . The computer-implemented method of claim 8 , wherein the other one of the possible OCR texts has a second highest corresponding OCR score relative to the OCR scores of the other possible OCR texts.

10 . A computer-implemented method, comprising:
receiving, at a computing system having one or more processors, a first image of an object comprising a text in a source language, the first image being captured by a camera in communication with the computing system; performing, by the computing system, optical character recognition (OCR) on the first image to obtain a set of possible OCR texts, wherein each possible OCR text has a corresponding OCR score indicative of a degree of likelihood that the possible OCR text is the text; selecting, by the computing system, the possible OCR text having a highest corresponding OCR score relative to the OCR scores of the other possible OCR texts to obtain an OCR text; and when the OCR score of the OCR text less than an OCR score threshold indicative of an acceptable degree of likelihood:
outputting, by the computing system, a user instruction to capture a set of second images of at least a portion of the object using the camera;
receiving, at the computing system, the set of second images;
performing, by the computing system, OCR on at least one of the set of second images to obtain a modified OCR text corresponding to the text; and
outputting, by the computing system, the modified OCR text.

11 . The computer-implemented method of claim 10 , further comprising capturing, by the computing system, the first image and the set of second images using the camera.

12 . The computer-implemented method of claim 11 , further comprising:
when the OCR score is less than the OCR score threshold, adjusting, by the computing system, an intensity of a light source associated with the camera while capturing the second set of images, the light source being in communication with the computing system.

13 . The computer-implemented method of claim 11 , further comprising:
when the OCR score is less than the OCR score threshold, adjusting, by the computing system, a number of flashes of a light source associated with the camera while capturing the second set of images, the light source being in communication with the computing system.

14 . The computer-implemented method of claim 10 , further comprising:
when the translation score is less than the translation score threshold, identifying, by the computing system, a portion of the first image causing the OCR score to be less than the OCR score threshold, wherein the user instruction is to capture the set of second images with respect to the identified portion of the first image.

15 . The computer-implemented method of claim 14 , wherein performing OCR on at least one of the set of second images to obtain a modified OCR text corresponding to the text comprises:
combining portions of at least two of the first image and the set of second images to obtain a combined image; and performing OCR on the combined image to obtain the modified OCR text.

16 . The computer-implemented method of claim 10 , further comprising when the OCR score is less than the OCR score threshold, selecting, by the computing system, another one of the possible OCR texts to obtain the modified OCR text.

17 . The computer-implemented method of claim 16 , wherein the other one of the possible OCR texts has a second highest corresponding OCR score relative to the OCR scores of the other possible OCR texts.

18 . The computer-implemented method of claim 10 , further comprising:
ob

Assignees

• Google Inc

Inventors

• Cuthbert Alexander Jay
• Hughes Macduff Richard

Classifications

• G06V30/127 Primary

  with the intervention of an operator · CPC title

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G06F40/58

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06F40/51

  Translation evaluation · CPC title

• G06F40/10

  Text processing (natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40) · CPC title