Electronic device for processing user utterance and control method thereof

Patent metadata

Field	Value
Publication number	US-12148425-B2
Application number	US-202217718970-A
Country	US
Kind code	B2
Filing date	Apr 12, 2022
Priority date	Apr 12, 2021
Publication date	Nov 19, 2024
Grant date	Nov 19, 2024

Abstract

An electronic device is provide that may identify voiceprint data corresponding to a user from among at least one piece of voiceprint data, based on received user voice data, identify a general voice instruction included in the received user voice data, determine user preference information of the user, based on the identified voiceprint data, determine a control action for determining an action to be performed in at least one external device or the electronic device, based on the general voice instruction identified from the received user voice data, determine a personalized voice instruction, based on at least one of the control action or the user preference information, and transmit, to the at least one external device, through the communication circuit, an audio signal corresponding to the personalized voice instruction to be output by the at least one external device, or output the audio signal through a speaker included in the electronic device.

First claim

What is claimed is:

1. An electronic device, comprising:
memory storing at least one piece of voiceprint data corresponding to each of at least one user;
communication circuitry configured to communicate with at least one external device; and
at least one processor electrically connected to the memory and the communication circuitry,
wherein the at least one processor is configured to
identify voiceprint data corresponding to the user from among the at least one piece of voiceprint data, based on received user voice data,
identify a general voice instruction included in the received user voice data,
determine user preference information of the user, based on the identified voiceprint data,
determine a control action for determining an action to be performed in the at least one external device or the electronic device, based on the general voice instruction identified from the received user voice data,
determine a personalized voice instruction, based on at least one of the control action or the user preference information, and
transmit, through the communication circuitry, to the at least one external device, an audio signal corresponding to the personalized voice instruction to be output by the at least one external device, or output the audio signal through a speaker included in the electronic device.

2. The electronic device of claim 1 , wherein the memory stores the control action in association with at least one of the personalized voice instruction or the user preference information, and
wherein the at least one processor is further configured to retrieve the personalized voice instruction stored in association with at least one of the control action or the user preference information in response determining the control action and the user preference information.

3. The electronic device of claim 1 , wherein the at least one processor is further configured to:
determine an additional control action, based on at least one of the control action, the user preference information, or device information of a first external device that is a target of the control action, and
control the memory to store the additional control action in association with the personalized voice instruction.

4. The electronic device of claim 3 , wherein the at least one processor is further configured to:
generate a control signal for controlling the first external device to perform the control action and the additional control action, and
transmit the control signal to the first external device through the communication circuitry.

5. The electronic device of claim 1 , wherein the at least one processor is further configured to:
store, in the memory, voice instruction history information and age information corresponding to each of the at least one user, or preference information input from each of the at least one user, and
determine the user preference information, based on at least one piece of voice instruction history information of the user among the at least one user, age information of the user, and preference information input from the user.

6. The electronic device of claim 3 , wherein the at least one processor is further configured to determine the additional control action, based on at least one of the control action, the device information of the first external device, a time at which the user voice data is received, or location information of the electronic device when the user voice data is received.

7. The electronic device of claim 1 , wherein the at least one processor is further configured to receive status information of an external device for each of the at least one external device using the communication circuitry.

8. The electronic device of claim 7 , wherein the at least one processor is further configured to:
determine whether an external device is reproducing content, among the at least one external device, based on the status information of the external device, and
control the output power of the audio signal, based on a result of the determination.

9. The electronic device of claim 7 , wherein the at least one processor is further configured to:
determine whether an external device is reproducing content, among the at least one external device, based on the status information of the external device,
determine a content voice instruction, based on the content being reproduced, and
transmit, through the communication circuitry, to the at least one external device, an audio signal corresponding to the content voice instruction to be output, or output the audio signal through the speaker, and
wherein the content voice instruction is determined based on information included in the content being reproduced.

10. The electronic device of claim 1 , wherein the at least one processor is further configured to:
determine a subsequent instruction to be used after using the general voice instruction, based on the general voice instruction,
determine a subsequent personalized instruction, based on the subsequent instruction and the user preference information, and
transmit, to the at least one external device, through the communication circuitry, a subsequent audio signal corresponding to the subsequent personalized instruction to be output by the at least one external device, or output the subsequent audio signal through the speaker, and
wherein the subsequent instruction includes an instruction spoken by the user after the general voice instruction.

11. The electronic device of claim 1 , wherein the at least one processor is further configured to:
determine whether the control action includes a restricted action requiring an authentication action for the user of the electronic device, and
in response to the control action including the restricted action as a result of the determination, control the speaker to output a notification regarding restriction of the action corresponding to the general voice instruction.

12. The electronic device of claim 1 , wherein the at least one processor is further configured to:
receive a user input for configuring a personalized voice instruction corresponding to a general voice instruction stored in the memory, and
control the memory to store the personalized voice instruction in association with the general voice instruction.

13. The electronic device of claim 1 , wherein the external device includes a display, and
wherein the processor is further configured to:
transmit, to an external device comprising a display, using the communication circuitry, a personalized instruction list corresponding to the general voice instruction, and
transmit a list output signal instructing the external device to display the personalized instruction list using the display.

14. A method performed by an electronic device including memory storing at least one piece of voiceprint data corresponding to at least one user and a communication circuitry configured to communicate with at least one external device, the method comprising:
receiving user voice data;
identifying voiceprint data corresponding to the user from among the at least one piece of voiceprint data, based on received user voice data;
identifying a general voice instruction included in the received user voice data;
determining user preference information of the user, based on the identified voiceprint data;
determining a control action based on an action to be performed in the at least one external device or the electronic device, based on the general voice instruction;
determining a personalized voice instruction, based on at least one of the control action or the user preference information; and
transmit

Assignees

• Samsung Electronics Co Ltd

Inventors

• Hong Yeseul
• Lee Dayoung
• Lee Boram

Classifications

• G10L25/78

  Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L2015/225

  Feedback of the input speech · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title