Electronic device for providing interaction on basis of user voice, and method therefor

Patent metadata

Field	Value
Publication number	US-12521887-B2
Application number	US-202318207989-A
Country	US
Kind code	B2
Filing date	Jun 9, 2023
Priority date	Dec 9, 2020
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

An electronic device can include a microphone; a camera module; a short-range communication module supporting short-range wireless communication; a communication module configured to communicate with a voice recognition server; a memory; and a processor. The processor may be configured to: identify whether an object accessing the electronic device is a user; determine whether a voice interaction condition is satisfied on the basis of context information; when the user's access is identified, if the voice interaction condition is satisfied, receive user voice from the microphone, and if the voice interaction condition is not satisfied, output external interaction information that enables an external electronic device to interact with the voice recognition server by using the short-range communication module; receive user voice analysis information from the voice recognition server by using the communication module; and perform at least one operation on the basis of the received user voice analysis information.

First claim

The invention claimed is:

1 . An electronic device comprising:
a microphone; a camera module; a short range communication module configured to support a short range wireless communication; a communication module configured to communicate with a voice recognition server;
a memory; and
a processor operatively connected to the microphone, the communication module, and the memory,
wherein the processor is configured to:
identify, using the camera module, whether an object that accesses the electronic device is a user,
identify, based on context information, whether a voice interaction condition is satisfied,
based on identifying that the object is the user and that the voice interaction condition is satisfied:
receive a user voice from the microphone, and
perform at least one operation based on the received user voice, and

based on the voice interaction condition not being satisfied:
output external interaction information enabling an external electronic device to perform an interaction with the voice recognition server by using the short range communication module,
receive, using the communication module, user voice analysis information from the voice recognition server,
perform at least one operation based on the received user voice analysis information,
determine whether an interaction end condition is satisfied, and
delete or deactivate the external interaction information based on the interaction end condition being satisfied;

wherein the external interaction information comprises a web page link related to the voice recognition server, and
wherein the web page link is deactivated when a specific time elapses after the external interaction information is output.

2 . The electronic device of claim 1 , further comprising a driving module operatively connected to the processor and configured to physically move or rotate the electronic device,
wherein the processor is further configured to:
based on identifying that the object is the user, identify an interaction location including at least one of a place and a direction suitable for the interaction with the user, and
guide the user toward the interaction location.

3 . The electronic device of claim 2 , wherein the processor is further configured to:
identify at least one of noise level information obtained based on an audio signal input through the microphone and density level information obtained based on an image obtained by capturing the surroundings of the electronic device obtained by using the camera module, and identify the interaction location based on at least one of the noise level information and the density level information.

4 . The electronic device of claim 2 , wherein the processor is configured to guide the user toward the interaction location through at least one of movement and rotation toward the interaction location by using the driving module.

5 . The electronic device of claim 2 , further comprising a speaker and a display module,
wherein the processor is further configured to:
guide the user toward an optimum interaction location by outputting a message for guiding the interaction location to at least one of the speaker and the display module.

6 . The electronic device of claim 1 , wherein the context information comprises at least one of noise level information, density level information, content sensitivity information, and user selection information.

7 . The electronic device of claim 1 , wherein the processor is further configured to:
based on the voice interaction condition not being satisfied, newly generate the external interaction information, and output the generated external interaction information.

8 . The electronic device of claim 1 , wherein the short range wireless communication is a near field communication (NFC) or an ultra-wideband (UWB) communication.

9 . The electronic device of claim 1 , further comprising a display module,
wherein the processor is further configured to generate and output a quick response (QR) code for the external interaction information to the display module.

10 . The electronic device of claim 1 , wherein the processor is further configured to:
recognize, using the camera module, different users, update the external interaction information based on the recognition result of the recognized different users, and output the updated external interaction information.

11 . The electronic device of claim 1 , wherein the interaction end condition is determined based on at least one of whether more than a predetermined time elapses from the time when the voice analysis information on the user having accessed the electronic device is finally received, whether the user having accessed the electronic device has moved beyond a predetermined distance from the electronic device, whether at least one operation based on the received user voice analysis information is ended in all, and whether the voice interaction condition is switched from an unsatisfied state to a satisfied state.

12 . The electronic device of claim 1 , wherein the processor is further configured to output, responsive to the voice interaction condition not being satisfied, a message for guiding the user's reception of the external interaction information to the speaker or the display module.

13 . A method, by an electronic device, for performing an interaction with a user, the method comprising:
identifying whether an object that accesses the electronic device is a user; identifying, based on context information, whether a voice interaction condition is satisfied; based on identifying that the object is the user and that the voice interaction condition is satisfied:
receiving a user voice; and
performing at least one operation based on the received user voice;
based on the voice interaction condition not being satisfied:
outputting external interaction information enabling an external electronic device to perform an interaction with a voice recognition server by using short range wireless communication;
receiving user voice analysis information from the voice recognition server;
performing at least one operation based on the received user voice analysis information;
determine whether an interaction end condition is satisfied; and
delete or deactivate the external interaction information based on the interaction end condition being satisfied;
recognizing different users; updating the external interaction information based on a recognition result of the recognized different users; and outputting the updated external interaction information.

14 . The method of claim 13 , further comprising:
based on identifying that the object is the user, identifying an interaction location including at least one of a place or a direction suitable for the interaction with the user, in case that the user's access is identified; and guiding the user toward the interaction location.

15 . The method of claim 13 , wherein the context information comprises at least one of noise level information, density level information, content sensitivity information, and user selection information.

16 . The method of claim 13 , wherein the short range wireless communication is a near field communication (NFC) or an ultra-wideband (UWB) communication,
wherein the external interaction information comprises a web page link related to the voice recognition server, and wherein the web page link is deactivated when a specific time elapses after the external interaction information is output.

17 . The method of claim 13 , wherei

Assignees

• Samsung Electronics Co Ltd

Inventors

• Cho Heeyoung
• Kim Harkjoon
• Park Jiwon
• Lee Hyeryoung

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L25/51

  for comparison or discrimination · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06K19/06037

  multi-dimensional coding · CPC title