Display device and method for operating same

Patent metadata

Field	Value
Publication number	US-12598342-B2
Application number	US-202418818361-A
Country	US
Kind code	B2
Filing date	Aug 28, 2024
Priority date	Dec 23, 2019
Publication date	Apr 7, 2026
Grant date	Apr 7, 2026

Abstract

A display device can include a display, a receiver to receive signals from a remote control including a microphone to receive a user voice command, and a controller operatively connected to the receiver. Also, the controller is configured to search for real-time information based on the user voice command including identifying information related to at least one of a person, a location, and an item. Also, the controller can acquire a captured image by capturing a content screen at a time point when the voice command is received while displaying a scene, identify a subject matter related to the voice command, the subject matter including information about a person, a location or an item in the captured image at the time point, search at least one database based on the subject matter to retrieve search results, and display the search results along with the captured image while displaying the scene.

First claim

The invention claimed is:

1 . A display device comprising:
a display configured to display content; a receiver configured to receive control signals from a remote control, the remote control including a push-to-talk microphone configured to receive a user voice command; and a controller operatively connected to the receiver, wherein the controller is configured to search for real-time information based on the user voice command, the user voice command including identifying information related to at least one of a person, a location, and an item; wherein the controller is further configured to: in response to receiving the user voice command while displaying a scene, capture a content screen image and determine a frame time corresponding to a time point when the user voice command is received, transmit a request including a content identifier and the frame time to a metadata server that stores content-related information categorized per frame time, receive, from the metadata server, category information corresponding to the frame time including at least one of person information, location information, and item information, identify a requested category from the user voice command and arrange the received category information so that information of the requested category is displayed with a first priority and information of other categories with a second priority, display the arranged category information together with the captured content screen image while continuing to display the scene, and in response to the requested category being absent for the frame time, obtain category information for a previous scene and display the category information corresponding to the requested category from the previous scene.

2 . The display device according to claim 1 , wherein the user voice command includes at least one of a who query related to a character in the scene, a where query related to a location in the scene, and a what query related to an item in the scene.

3 . The display device according to claim 1 , wherein the controller is further configured to:
display the captured content screen image at an upper left corner of the display together with the search results while continuing to display the scene at a right side of the display, and display the search results under the captured content screen image.

4 . The display device according to claim 3 , wherein the controller is further configured to:
display a plurality of thumbnail images corresponding to the search results at a location under the captured content screen image, and display arrow buttons under the plurality of thumbnail images for scrolling through the search results.

5 . The display device according to claim 4 , wherein the controller is further configured to:
identify a category topic from the user voice command, and display the category topic above the plurality of thumbnail images corresponding to the search results, wherein the category topic is based on the at least one subject matter related to the user voice command.

6 . The display device according to claim 1 , wherein the category topic includes at least one of a location category corresponding to the scene, a product category corresponding to the scene, a person category corresponding to the scene, and an item category corresponding to the scene.

7 . The display device according to claim 1 , wherein the controller is further configured to:
display the scene across an entire screen area of the display, and in response to receiving the user voice command, divide a screen area of the display into a left side and a right side, and display the search results along with the captured content screen image on the left side of the screen area and display the scene on the right side of the screen area.

8 . The display device according to claim 7 , wherein the controller is further configured to:
in response to no category topic being included in the user voice command, display the search results on the left side according to a default priority order.

9 . The display device according to claim 8 , wherein the default priority order includes an order of location information, person information, and item information.

10 . The display device according to claim 7 , wherein the controller is further configured to:
display other items in the screen on the left side, the other items being unrelated to a category topic identified from the user voice command.

11 . The display device according to claim 1 , wherein the controller is further configured to:
in response to receiving the user voice command, display a view all icon, in response to a selection of the view all icon, display all information included in a plurality of category information corresponding to the scene, the all information including one or more of character information corresponding to the scene, item information corresponding to scene product information corresponding to the scene, location information corresponding to the scene, and music information corresponding to the scene.

12 . The display device according to claim 1 , wherein the controller is further configured to:
receive the user voice command, in response to a press of a button on the remote control.

13 . The display device according to claim 1 , wherein the controller is further configured to:
determine a requested category based on the user voice command, in response to no information being available for the requested category, display other information different than the requested category.

14 . The display device according to claim 1 , wherein the controller is further configured to:
determine a requested category based on the user voice command, in response to no information being available for the requested category based on the scene, display search results corresponding to information related to a previous scene that was displayed before receiving the user voice command.

15 . The display device according to claim 1 , wherein the controller is further configured to:
determine a requested category based on the user voice command, in response to no information being available for the requested category based on the scene, display a message overlaid on the scene indicating that no information is available for the requested category.

16 . A display device comprising:
a display configured to display content; and a controller configured to: in response to receiving a voice command including a requested category while displaying a scene, obtain a captured still image of the scene, determine a frame time corresponding to a time point of the voice command, and query a metadata server that stores content-related information categorized per frame time to receive category information corresponding to the frame time, and display category information arranged according to the requested category along with the captured still image while continuing to display the scene, including obtaining information for a previous scene when the requested category is absent for the frame time.

17 . The display device according to claim 16 , wherein the controller is further configured to:
in response to receiving the voice command, divide a screen of the display into a first area and a second area, and display search results along with the captured still image on the first area of the screen and display the scene in the second area of the screen.

18 . The display device according to claim 17 , wherein the controller is further configured to:
display the capture

Assignees

• Lg Electronics Inc

Inventors

• Lee Hyeseung
• Cho Eunyoung

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• H04N21/4394

  involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58) · CPC title

• H04N21/4622

  Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859) · CPC title