Electronic device and control method thereof

Patent metadata

Field	Value
Publication number	US-11889152-B2
Application number	US-202017609669-A
Country	US
Kind code	B2
Filing date	Jun 24, 2020
Priority date	Nov 27, 2019
Publication date	Jan 30, 2024
Grant date	Jan 30, 2024

Abstract

An electronic device and a control method thereof are provided. The electronic device includes a camera, a microphone, a display, a communicator including circuitry, a memory, and a processor configured to capture an image of an external device through the camera and obtain information on the external device, based on a user's voice for requesting for solution information for solving a problem regarding the external device being input through the microphone while obtaining the information on the external device, control the communicator to transmit the input user's voice and the information on the external device to a server, receive a content related to the solution information from a content of a device related to the external device from the server via the communicator, and control the display to provide the content related to the solution information along with the captured image.

First claim

What is claimed is:

1. An electronic device comprising:
a camera;
a microphone;
a display;
a communicator comprising circuitry;
a memory; and
a processor configured to:
capture an image of an external device through the camera,
capture, through the microphone, a user's voice for requesting solution information for solving a problem regarding the external device,
based on the user's voice, control the communicator to transmit a message comprising the user's voice and the image of the external device to a server,
receive a content related to the solution information from a content of a device related to the external device from the server via the communicator, the content related to the solution information comprising a region on the external device for a next step for solving the problem, and
control the display to provide the content related to the solution information along with the captured image,

wherein the device related to the external device is identified based on the image of the external device,
wherein the content related to the solution information comprises a text corresponding to the solution information from a manual of the device related to the external device, and
wherein the processor is further configured to identify whether the region exists in the image of the external device based on the text.

2. The device of claim 1 , wherein the content related to the solution information comprises at least one of the manual of the device related to the external device, a video content related to the solution information from a guide video content of the device related to the external device, or a subtitle of the video content related to the solution information.

3. The device of claim 1 , wherein the processor is further configured to, based on identification that the region exists in the image of the external device, control the display to display the text on the region.

4. The device of claim 1 , wherein the processor is further configured to:
based on identification that the region exists in the image of the external device, obtain a subtitle related to the text from the subtitle of video content related to the solution information, and
identify a point when the obtained subtitle is displayed on the video content.

5. The device of claim 4 , wherein the processor is further configured to control the display to display at least one of a graphical user interface (GUI) or a hyper link for viewing a video content corresponding to the point from the video content on the region.

6. The device of claim 5 , wherein the processor is further configured to, based on at least one of the GUI or the hyper link being selected, control the display to display the video content corresponding to the point on the region.

7. The device of claim 1 , wherein the processor is further configured to, based on identification that the region does not exist in the image of the external device, provide a message instructing to capture the region.

8. A server comprising:
a communicator comprising circuitry;
a memory; and
a processor configured to:
receive a message comprising a captured user's voice for requesting solution information for solving a problem regarding an external device and an image of the external device from an electronic device via the communicator,
identify a device related to the external device based on the image of the external device,
obtain a content related to the solution information from a content of the identified device related to the external device, the content related to the solution information comprising a region on the external device for a next step for solving the problem, and
control the communicator to transmit the content related to the solution information to the electronic device,

wherein the content related to the solution information comprises a text corresponding to the solution information from a manual of the device related to the external device, and
wherein the electronic device identifies whether the region exists in the image of the external device based on the text.

9. A method for controlling an electronic device, the method comprising:
capturing an image of an external device through a camera;
capturing, through a microphone, a user's voice for requesting solution information for solving a problem regarding the external device,
based on the user's voice, transmitting a message comprising the user's voice and the image of the external device to a server;
receiving a content related to the solution information from a content of a device related to the external device from the server, the content related to the solution information comprising a region on the external device for a next step for solving the problem; and
providing the content related to the solution information along with the captured image,
wherein the device related to the external device is identified based on the image of the external device,
wherein the content related to the solution information comprises a text corresponding to the solution information from a manual of the device related to the external device, and
wherein the method further comprises identifying whether the region exists in the image of the external device based on the text.

10. The method of claim 9 , wherein the content related to the solution information comprises at least one of the manual of the device related to the external device, a video content related to the solution information from a guide video content of the device related to the external device, or a subtitle of the video content related to the solution information.

11. The method of claim 9 , wherein the providing of the content comprises, based on identification that the region exists in the image of the external device, displaying the text on the region.

12. The method of claim 9 , wherein the providing of the content comprises:
based on identification that the region exists in the image of the external device, obtaining a subtitle related to the text from the subtitle of video content related to the solution information; and
identifying a point when the obtained subtitle is displayed on the video content.

13. The method of claim 12 , wherein the providing of the content comprises displaying at least one of a graphical user interface (GUI) or a hyper link for viewing a video content corresponding to the point from the video content on the region.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Ham Jina
• Choi Wonjong
• Kim Soofeel
• Park Yewon

Classifications

• H04N21/47202 Primary

  for requesting content on demand, e.g. video on demand · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• H04N21/42203 Primary

  sound input device, e.g. microphone · CPC title

• H04N21/4884

  for displaying subtitles · CPC title

• H04N7/18

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title