Language translation device and language translation method

Patent metadata

Field	Value
Publication number	US-10643036-B2
Application number	US-201715681221-A
Country	US
Kind code	B2
Filing date	Aug 18, 2017
Priority date	Aug 18, 2016
Publication date	May 5, 2020
Grant date	May 5, 2020

Abstract

A method of providing real-time translation for video chat is provided. The method includes: continuously receiving first-language voice data and at least one second-language word from a first terminal; continuously displaying the at least one second-language word at the same time as reproduction of the voice data; acquiring a second-language translation of an ended sentence included in a voice recognition result for the voice data; and substituting at least one word, which corresponds to the ended sentence in the displayed at least one second-language word, with the acquired translation. The at least one second-language word corresponds to respective words included in the voice recognition result for the voice data.

First claim

What is claimed is:

1. A method of a video call between a first terminal of a first user speaking a first language and a second terminal of a second user speaking a second language, the method comprising:
displaying, on a display screen of the first terminal, a video stream that is received from the second terminal and comprises the second user's verbal expression in the second language of a sentence comprising a first word, a second word and a third word in order and ending with the third word; and
displaying, on the display screen of the first terminal, a sequence of visual representations in the first language along with the video stream,
wherein displaying the sequence of visual representations comprises:
displaying a first visual representation in response to the first word of the second user's verbal expression, wherein the first visual representation comprises a translation of the first word in the first language on the display screen,
subsequently, displaying, on the display screen, a second visual representation in response to the second word of the second user's verbal expression, wherein the second visual representation comprises the translation of the first word and a translation of the second word in the first language in order,
further subsequently, displaying, on the display screen, a third visual representation in response to the third word of the second user's verbal expression, wherein the third visual representation comprises the translation of the first word, the translation of the second word, and a translation of the third word in the first language in order, and
further subsequently, in response to detecting an end point of the second user's verbal expression of the sentence, displaying a translation in the first language of the sentence of the second user's verbal expression in replacement of the third visual representation such that the third visual representation disappears from the display screen and the translation of the sentence is presented on the display screen.

2. The method of claim 1 , wherein the sentence is referred to as a first sentence and the video stream further comprises the second user's verbal expression in the second language of a second sentence following the first sentence, the first sentence ending with the third word, the second sentence comprising a fourth word following the third word of the first sentence,
wherein a translation of the fourth word in the first language is presented prior to presentation of the translation of the first sentence such that the translation of the first word, the translation of the second word, the translation of the third word, and the translation of the fourth word are presented in order on the display.

3. The method of claim 1 , further comprising processing the second user's verbal expression to detect the end point based on at least one of an intonation in the second user's verbal expression and a blank period of the second user's verbal expression.

4. The method of claim 3 , wherein the end point is determined based on at least one of an intonation in the second user's verbal expression and a blank period of the second user's verbal expression.

5. The language translation method of claim 1 , wherein the translation of the first word is presented before completion of the second user's verbal expression of the sentence.

6. The method of claim 5 , wherein the translation of the second word is presented before completion of the second user's verbal expression of the sentence.

7. The method of claim 6 , wherein the translation of the third word is presented before completion of the second user's verbal expression of the sentence.

8. A non-transitory computer-readable recording medium having recorded thereon a program for executing the language translation method of claim 1 .

Assignees

• Hyperconnect Inc

Inventors

• Ahn Sangil
• Jung Kangsik
• Yong Hyountaek
• Choi Hyeok

Classifications

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• G06F40/51

  Translation evaluation · CPC title

• G06F40/42

  Data-driven translation · CPC title

• G10L15/26 Primary

  Speech to text systems (G10L15/08 takes precedence) · CPC title