Electronic device and control method thereof

Patent metadata

Field	Value
Publication number	US-11227608-B2
Application number	US-202016899854-A
Country	US
Kind code	B2
Filing date	Jun 12, 2020
Priority date	Jan 23, 2020
Publication date	Jan 18, 2022
Grant date	Jan 18, 2022

Abstract

An electronic device is provided. The electronic device includes a memory storing recording data including a content of a conversation and at least one instruction, and a processor configured, by executing the at least one instruction, to input first data corresponding to a first voice in the content of the conversation into a first neural network model and acquire category information of the first data, and acquire category information of second data corresponding to a second voice in the content of the conversation. The processor is configured to, based on the category information of the first data and the category information of the second data being different, train the first neural network model based on the category information of the second data and the first data.

First claim

What is claimed is:

1. An electronic device comprising:
a memory storing recording data including a content of a conversation and at least one instruction; and
a processor configured, by executing the at least one instruction, to:
input first data corresponding to a first voice in the content of the conversation into a first neural network model and acquire category information of the first data, and
acquire category information of second data corresponding to a second voice in the content of the conversation, and
based on the category information of the first data and the category information of the second data being different, train the first neural network model based on the category information of the second data and the first data.

2. The electronic device of claim 1 ,
wherein the memory stores voice profile information related to data corresponding to the second voice, and
the processor is configured to:
identify the second voice in the recording data based on the stored voice profile information.

3. The electronic device of claim 1 ,
wherein the processor is configured to:
input data corresponding to the second voice into a second neural network model and acquire the category information of the data corresponding to the second voice.

4. The electronic device of claim 3 ,
wherein the processor is configured to:
input a sentence acquired by combining at least a portion of each of a plurality of sentences included in the data corresponding to the second voice into the second neural network model and acquire the category information of the data corresponding to the second voice.

5. The electronic device of claim 4 ,
wherein the processor is configured to:
sequentially input text data corresponding to each of the plurality of sentences included in the data corresponding to the second voice into the second neural network model, and acquire at least one category information corresponding to each of the plurality of sentences and a first probability value corresponding to the at least one category information from the second neural network model,
input text data corresponding to the sentence acquired by combining at least a portion of each of the plurality of sentences into the second neural network model, and acquire at least one category information corresponding to the acquired sentence and a second probability value corresponding to the at least one category information from the second neural network model, and
train the first neural network model based on a sentence selected based on the first probability value and the second probability value and the first category information corresponding to the selected sentence.

6. The electronic device of claim 5 ,
wherein the processor is configured to:
select a sentence having a probability value greater than or equal to a threshold value between the first probability value and the second probability value, and train the first neural network model based on the selected sentence and the first category information corresponding to the selected sentence.

7. The electronic device of claim 5 ,
wherein the processor is configured to:
based on a connective word being included in the sentence acquired by combining at least a portion of each of the plurality of sentences, apply a weight value to a probability value corresponding to the acquired sentence based on at least one of the type or number of times of the connective word.

8. The electronic device of claim 3 ,
wherein the processor is configured to:
based on acquiring at least one of moving image data, text data, or image data related to the second voice from at least one of moving image data, image data, or text data, input the acquired data into the second neural network model and acquire the category information of the data corresponding to the second voice.

9. The electronic device of claim 1 ,
wherein the processor is configured to:
based on user voice data being input, input text data corresponding to the user voice data into the trained first neural network model, and acquire category information corresponding to the user voice data, and acquire response information corresponding to the user voice data based on the category information.

10. The electronic device of claim 1 ,
wherein the recording data comprises data including a content of counseling between a customer and a counselor, and
the first voice is the voice of the customer, and
the second voice is the voice of the counselor.

11. A method of controlling an electronic device storing recording data including a content of a conversation, the method comprising:
inputting first data corresponding to a first voice in the content of the conversation into a first neural network model and acquiring category information of the first data;
acquiring category information of second data corresponding to a second voice in the content of the conversation; and
based on the category information of the first data and the category information of the second data being different, training the first neural network model based on the category information of the second data and the first data.

12. The method of claim 11 , further comprising:
identifying the second voice in the recording data based on voice profile information related to the pre-stored data corresponding to the second voice.

13. The method of claim 11 ,
wherein the acquiring the category information of the second data comprises:
inputting data corresponding to the second voice into a second neural network model and acquiring the category information of the data corresponding to the second voice.

14. The method of claim 13 ,
wherein the acquiring the category information of the second data comprises:
inputting a sentence acquired by combining at least a portion of each of a plurality of sentences included in the data corresponding to the second voice into the second neural network model and acquiring the category information of the data corresponding to the second voice.

15. The method of claim 14 ,
wherein the training the first neural network model comprises:
sequentially inputting text data corresponding to each of the plurality of sentences included in the data corresponding to the second voice into the second neural network model, and acquiring at least one category information corresponding to each of the plurality of sentences and a first probability value corresponding to the at least one category information from the second neural network model;
inputting text data corresponding to the sentence acquired by combining at least a portion of each of the plurality of sentences into the second neural network model, and acquiring at least one category information corresponding to the acquired sentence and a second probability value corresponding to the at least one category information from the second neural network model; and
training the first neural network model based on a sentence selected based on the first probability value and the second probability value and the first category information corresponding to the selected sentence.

16. The method of claim 15 ,
wherein the training the first neural network model comprises:
selecting a sentence having a probability value greater than or equal to a threshold value between the first probability value and the second probability value, and training the first neural network model based on the selected sentence and the category information corresponding to the selected sentence.

17. The method of claim 15 , further comprising:
based on a connective word being i

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Soofeel
• Ham Jina
• Park Yewon
• Choi Wonjong

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L17/00

  Speaker identification or verification techniques · CPC title