Terminal apparatus and method for controlling internet of things (IoT) devices

Patent metadata

Field	Value
Publication number	US-11329938-B2
Application number	US-201616088178-A
Country	US
Kind code	B2
Filing date	Dec 23, 2016
Priority date	Mar 31, 2016
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

An example terminal apparatus includes a communicator configured to communicate with at least one device through a hub device; a display configured to display an image; a user input receiver configured to receive a user's input; and at least one processor configured to control the display to display a dialog box where the hub device or the at least one device is set as a counterpart in response to execution of a messenger program, control the communicator to transmit a control signal including a command for controlling a predetermined device to the device targeted for control through the hub device so that the device targeted for control operates in response to the command corresponding to a user's input, and control the display to display a message corresponding to the transmitted command in the dialog box.

First claim

What is claimed is:

1. A terminal apparatus comprising:
a communicator, comprising a communication circuit, configured to communicate with a hub device and at least one device through the hub device;
a display configured to display an image;
an input receiver configured to receive input and comprising at least one of a button, a microphone, a keypad, or an input device; and
at least one processor configured to:
based on a messenger program being executed in the terminal apparatus according to an input through the input receiver, control the display to display a first user interface (UI) comprising a first item of a chat user in a first area of the display, a message input window in a second area of the display, and a message displaying window in a third area of the display,
based on the first item being selected at the first area and a message being input into the message input window of the second area in the displayed first UI through the input receiver, control the communicator to transmit the input message to a first device of the chat user and control the display to display the transmitted message in the message displaying window of the third area of the display,
based on a first tab of the displayed first UI being selected through the input receiver, control the display to display a second UI comprising a second item corresponding to a first command for controlling a second device in the first area, while maintaining the message input window in the second area and the message displaying window in the third area, by replacing the first item with the second item in the first area of the display, and
based on the second item being selected in the displayed second UI through the user input receiver, control the communicator to transmit a first control signal including the first command to the hub device, which identifies the second device corresponding to the first command and transmits the first control signal thereto, or to transmit the first control signal to the second device through the hub device, and control the display to display a message corresponding to the first command in the message displaying window of the third area of the display,

wherein the at least one processor is further configured to:
based on the second item being selected in the displayed second UI, control the display to display a drawing area capable of receiving a drawing input for the first command in the first area of the display,
based on the message input window being selected in the displayed second UI, control the display to display the keypad capable of selecting a text for the first command in the first area of the display, and
based on a voice input icon being selected in the displayed second UI, activate the microphone to sense a voice input for the first command.

2. The terminal apparatus according to claim 1 , wherein the at least one processor is further configured to:
control the display to display a plurality of selectable icons corresponding to commands for controlling the one or more devices, and
the first command corresponds to an icon selected in response to an input through the input receiver.

3. The terminal apparatus according to claim 2 , wherein the plurality of selectable icons is displayed in response to selection of the first tab, and the first tab is included in the keypad.

4. The terminal apparatus according to claim 3 , wherein the first tab is activated in response to an input for registering the hub device or the at least one device.

5. The terminal apparatus according to claim 4 , wherein the processor is configured to automatically register the hub device or the at least one device in the messenger program or control the display to display information about the hub device or the at least one device in a recommendation list.

6. The terminal apparatus according to claim 2 , wherein at least one icon among the plurality of selectable icons corresponds to a device targeted for control, and
the processor is configured to receive, via the input receiver, a first input for selecting one among the plurality of selectable icons, and a second input corresponding to a command for controlling a device corresponding to the selected icon.

7. The terminal apparatus according to claim 2 , wherein the processor is configured to:
receive, via the input receiver, an input for generating a group comprising a plurality of devices of the at least one device, and
control the communicator to transmit a second control signal comprising a second command to the plurality of devices through the hub device so that the plurality of devices included in the group is controlled based on the second command.

8. The terminal apparatus according to claim 2 , wherein the first control signal comprising the first command is transmitted to the hub device via a communication network or mobile communication network of a service provider of the messenger program.

9. A method of controlling a terminal apparatus, the method comprising:
based on a messenger program being executed in the terminal apparatus according to an input to the terminal apparatus, displaying a first user interface (UI) comprising a first item of a chat user in a first area of a display of the terminal apparatus, a message input window in a second area of the display, and a message displaying window in a third area of the display;
based on a first item being selected at the first area and a message being input into the message input window of the second area in the displayed first UI, transmitting the input message to a first device of the chat user and displaying the transmitted message in the message displaying window of the third area of the display;
based on a first tab of the displayed first UI being selected, displaying a second UI comprising a second item corresponding to a first command for controlling a second device in the first area, while maintaining the message input window in the second area and the message displaying window in the third area, by replacing the first item with the second item in the first area of the display; and
based on the second item being selected in the displayed second UI, transmitting a first control signal including the first command to a hub device, which identifies the second device corresponding to the first command and transmits the first control signal thereto, or transmitting the first control signal to the second device through the hub device, and displaying a message corresponding to the first command in the message displaying window of the third area of the display,
wherein the displaying the second UI further comprises:
based on the second item being selected in the displayed second UI, displaying a drawing area capable of receiving a drawing input for the first command in the first area of the display,
based on the message input window being selected in the displayed second UI, displaying a keypad capable of selecting a text for the first command in the first area of the display, and
based on a voice input icon being selected in the displayed second UI, activating a microphone to sense a voice input for the first command.

10. The method according to claim 9 , further comprising:
displaying a plurality of selectable icons corresponding to commands for controlling the at least one device, wherein
the first command corresponds to an icon selected in response to an input to the terminal apparatus.

11. The method according to claim 10 , further comprising displaying the plurality of selectable icons in response to selection of the first tab.

12. The method according to claim 11 , further comprising activating the first tab in response to an input for registering t

Assignees

• Samsung Electronics Co Ltd

Inventors

• Dinhthi Thuy Nga
• Lanka Madan Kanth
• Dayama Dwarka Prasad
• Jung Markus
• Channa Ashok Babu
• Jampani Naga Ashok Babu
• Ha Ji-Hun
• Youn So-Young

Classifications

• H04L12/2818 Primary

  from a device located outside both the home and the home network (access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; arrangements for transmitting signals characterised by the use of a wireless eletrical link G08C17/00; telephonic communication systems adapted for combination with remote control systems H04M11/007) · CPC title

• G06F3/04817

  using icons (graphical or visual programming using iconic symbols G06F8/34) · CPC title

• G16Y10/75

  Information technology; Communication · CPC title

• H04M1/72415 Primary

  for remote control of appliances · CPC title

• H04M2250/06

  including a wireless LAN interface · CPC title