Electronic device and control method thereof

Patent metadata

Field	Value
Publication number	US-12211510-B2
Application number	US-202117551421-A
Country	US
Kind code	B2
Filing date	Dec 15, 2021
Priority date	Jan 23, 2020
Publication date	Jan 28, 2025
Grant date	Jan 28, 2025

Abstract

An electronic device is provided. The electronic device includes a memory storing recording data including a content of a conversation and at least one instruction, and a processor configured, by executing the at least one instruction, to input first data corresponding to a first voice in the content of the conversation into a first neural network model and acquire category information of the first data, and acquire category information of second data corresponding to a second voice in the content of the conversation. The processor is configured to, based on the category information of the first data and the category information of the second data being different, train the first neural network model based on the category information of the second data and the first data.

First claim

What is claimed is:

1. An electronic device comprising:
a memory storing recording data including a content of a conversation between a customer user and a counselor user, the conversation including a first voice corresponding to the customer user and a second voice corresponding to the counselor user, and at least one instruction; and
at least one processor configured, by executing the at least one instruction, individually and/or collectively, to:
input first data corresponding to the first voice in the content of the conversation into a neural network model and acquire first text corresponding to the first data, and category information of the first data identified from content of the first text output by the neural network model,
input second data corresponding to the second voice in the content of the conversation into the neural network model and acquire second text corresponding to the second data and category information of the second data identified from content of the second text output by the neural network model,
select first selected text from the first text and second selected text from the second text corresponding to at least one form of content from among a plurality of predefined forms, wherein at least a portion of the first text is excluded from the first text and at least a portion of the second text is excluded from the selected second text, and
train the neural network model based on the category information of the first data, the category information of the second data, the first selected text, and the second selected text,
wherein the category information of the first data and the category information of the second data are different,
wherein the category information of the first data and the category information of the second data are classified according to a predetermined depth,
wherein the first voice includes a customer inquiry corresponding to the customer user,
wherein the second voice is uttered after the first voice, and

wherein the second voice includes a response of the customer inquiry corresponding to the counselor user,
wherein the at least one processor is configured to:
sequentially input text data corresponding to each of the plurality of sentences included in the data corresponding to the second voice into the neural network model, and acquire at least one category information corresponding to each of the plurality of sentences and a first probability value corresponding to the at least one category information from the neural network model,
input text data corresponding to the sentence acquired by combining at least a portion of each of the plurality of sentences into the neural network model, and acquire at least one category information corresponding to the acquired sentence and a second probability value corresponding to the at least one category information from the neural network model, and
train the neural network model based on a sentence selected based on the first probability value and the second probability value and the category information corresponding to the selected sentence.

2. The electronic device of claim 1 , wherein the memory stores voice profile information related to data corresponding to the second voice, and
the at least one processor is configured to: identify the second voice in the recording data based on the stored voice profile information.

3. The electronic device of claim 1 , wherein the neural network model comprises a first neural network model and a second neural network model, and the at least one processor is configured to:
input the first data corresponding to the first voice into the first neural network model and acquire the category information of the first data based on output of the first neural network; and
input the second data corresponding to the second voice into the second neural network model and acquire the category information of the second data based on output of the second neural network.

4. The electronic device of claim 1 , wherein the category information of the first data and the category information of the second data is selected from a plurality of predefined categories.

5. The electronic device of claim 1 , wherein the at least one processor is configured to: train the neural network model based on the category information of the first data, the category information of the second data, the first text corresponding to the first data and the second text corresponding to the first data.

6. The electronic device of claim 3 , wherein the at least one processor is configured to:
based on acquiring at least one of moving image data, text data, or image data related to the second voice from at least one of moving image data, image data, or text data, input the acquired data into the second neural network model and acquire the category information of the data corresponding to the second voice.

7. The electronic device of claim 1 , wherein the neural network model includes a voice recognition model configured to recognize the first voice in the content and output the first text corresponding to the first data.

8. The electronic device of claim 1 , wherein the second selected text includes a plurality of sentences and the at least one processor is configured to:
input text data corresponding to each of the plurality of sentences into the neural network model, and acquire at least one first category information corresponding to each of the plurality of sentences and a first probability value corresponding to the at least one first category information from the neural network model,
input text data corresponding to a combined sentence acquired by combining at least a portion of each of the plurality of sentences into the neural network model, and acquire at least one second category information corresponding to the acquired sentence and a second probability value corresponding to the at least one second category information from the neural network model, and
train the neural network model based on the first probability value and the second probability value.

9. A method of controlling an electronic device storing recording data including a content of a conversation between a customer user and a counselor user, the conversation including a first voice corresponding to the customer user and a second voice corresponding to the counselor user, the method comprising:
inputting first data corresponding to the first voice in the content of the conversation into a neural network model and acquiring first text corresponding to the first data and category information of the first data identified from content of the first text output by the neural network model;
inputting second data corresponding to the second voice in the content of the conversation into the neural network model and acquiring second text corresponding to the second data and category information of the second data identified from content of the second text output by the neural network model;

select first selected text from the first text and second selected text from the second text corresponding to at least one form of content from among a plurality of predefined forms, wherein at least a portion of the first text is excluded from the first text and at least a portion of the second text is excluded from the selected second text; and
training the neural network model based on the category information of the first data, the category information of the second data, and the first selected text, and the second selected text corresponding to the second data,
wherein the category information of the first data and the category information of the second data are different,
wherein the category information of the first data and the category information

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Soofeel
• Ham Jina
• Park Yewon
• Choi Wonjong

Classifications

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G10L17/00

  Speaker identification or verification techniques · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F40/35

  Discourse or dialogue representation · CPC title