Language translation device and language translation method

Patent metadata

Field	Value
Publication number	US-11227129-B2
Application number	US-202016866406-A
Country	US
Kind code	B2
Filing date	May 4, 2020
Priority date	Aug 18, 2016
Publication date	Jan 18, 2022
Grant date	Jan 18, 2022

Abstract

A method of providing real-time translation for video chat is provided. The method includes: continuously receiving first-language voice data and at least one second-language word from a first terminal; continuously displaying the at least one second-language word at the same time as reproduction of the voice data; acquiring a second-language translation of an ended sentence included in a voice recognition result for the voice data; and substituting at least one word, which corresponds to the ended sentence in the displayed at least one second-language word, with the acquired translation. The at least one second-language word corresponds to respective words included in the voice recognition result for the voice data.

First claim

What is claimed is:

1. A language translation method at a transmitting terminal, the method comprising:
continuously receiving a stream of voice in a first-language;
continuously acquiring a voice recognition result for the stream of voice;
continuously acquiring second-language words, at least one of the second-language words being converted based on each of first-language words included in the voice recognition result;
continuously transmitting the acquired second-language words to a receiving terminal;
identifying an ended sentence included in the voice recognition result;
acquiring a second-language translation of the identified ended sentence, the identified ended sentence including the first-language words included in the voice recognition result; and
transmitting the acquired second-language translation to the receiving terminal to substitute at least one word of the transmitted second-language words for being displayed on the receiving terminal with at least a portion of the transmitted second-language translation.

2. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:
acquiring a similarity between a word order of the first-language and a word order of the second-language; and
when the acquired similarity is greater than a reference value, continuously converting each first-language word included in the voice recognition result into a second-language word having an identical or similar meaning thereto, and
wherein the continuously transmitting to the receiving terminal includes, when the acquired similarity is greater than the reference value, continuously transmitting the stream of voice and the acquired second-language words to the receiving terminal.

3. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:
transmitting the first-language words included in the voice recognition result to a first server; and
receiving the at least one second-language word having an identical or similar meaning to each of the transmitted first-language words from the first server.

4. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:
retrieving each of the first-language words included in the voice recognition result from a second database; and
when the second-language words corresponding to the retrieved first-language word is present in the second database, acquiring the second-language word corresponding to the retrieved first-language word from the second database as a result of the retrieving.

5. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:
retrieving one of the first language words included in the voice recognition result from a second database;
when a second-language word corresponding to the retrieved first-language word is not present in the second database, transmitting the retrieved first-language word to a second server;
receiving a second-language word having an identical or similar meaning to the transmitted first-language word from the second server; and
caching the transmitted retrieved first-language word and the received second-language word in the second database.

6. The language translation method of claim 1 , wherein the ended sentence included in the voice recognition result is identified based on
an end point using an intonation included in the received voice, a blank period included in the received voice, or a sentence constituent of the first-language word included in the voice recognition result; and
wherein the ended sentence is defined based on, section from a portion subsequent to a portion corresponding to a previous end point to a portion corresponding to the identified end point in the voice recognition result.

7. The language translation method of claim 6 , wherein the end point is identified based on an intonation corresponding to an end of a sentence in the received voice, a blank period exceeding a specific time in the received voice, or whether the voice recognition result includes first-language words corresponding to all sentence constituents necessary for constituting a full sentence.

8. The language translation method of claim 1 , wherein the acquiring of the second-language translation comprises:
transmitting the identified ended sentence to a first server; and
receiving the second-language translation of the ended sentence from the first server.

9. The language translation method of claim 1 , wherein the acquiring of the second-language translation comprises:
retrieving the identified ended sentence from a first database; and
when the second-language translation corresponding to the ended sentence is present in the first database, acquiring the second-language translation corresponding to the ended sentence from the first database as a result of the retrieving.

10. The language translation method of claim 1 , wherein the acquiring of the second-language translation comprises:
retrieving the identified ended sentence from a first database;
when the second-language translation corresponding to the ended sentence is not present in the first database, transmitting the ended sentence to a first server;
receiving the second-language translation of the ended sentence from the first server; and
caching the ended sentence and the received second-language translation of the ended sentence in the first database.

11. A terminal for transmitting information related to language translation, the terminal comprising;
a communication interface;
a display; and
a controller in data communication with the communication interface and the display, the controller configured to:
continuously receive a stream of voice in a first-language;
continuously acquire a voice recognition result for the stream of voice;
continuously acquire second-language words, at least one of the second-language words being converted based on each of first-language words included in the voice recognition result;
continuously transmit the acquired second-language words to a receiving terminal;
identify an ended sentence included in the voice recognition result;
acquire a second-language translation of the identified ended sentence, the identified ended sentence including the first-language words included in the voice recognition result; and
transmit the acquired second-language translation to the receiving terminal to substitute at least one word of the transmitted second-language words for being displayed on the receiving terminal with at least a portion of the transmitted second-language translation.

Assignees

• Hyperconnect Inc

Inventors

• Ahn Sangil
• Jung Kangsik
• Yong Hyountaek
• Choi Hyeok

Classifications

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06F9/454

  Multi-language systems; Localisation; Internationalisation · CPC title

• G10L15/005

  Language recognition · CPC title

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G06F40/55 Primary

  Rule-based translation · CPC title