Mobile terminal-based virtual game controller and remote control system using the same

Patent metadata

Field	Value
Publication number	US-9612709-B2
Application number	US-201313924186-A
Country	US
Kind code	B2
Filing date	Jun 21, 2013
Priority date	Jun 22, 2012
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

Disclosed herein are a virtual controller client mobile terminal-based virtual game controller and a remote control system using the same. The remote control system includes a virtual controller server and a virtual controller client. The virtual controller server generates button setting information including mapping relationship between key inputs and virtual input messages, transfers the button setting information, extracts a key input from a virtual input message, and provides the key input to the application. The virtual controller client specifies an arrangement and attributes of virtual buttons based on the button setting information, generates a virtual button screen on the touch screen of the mobile terminal, generates a touch input message based on touch event objects generated based on touch signals for regions corresponding to the virtual buttons, and converts the touch input message into a virtual input message and outputs the virtual input message.

First claim

What is claimed is:

1. A mobile terminal comprising a virtual controller client configured to remotely communicate with a virtual controller server running on a computer for remote key input to an application running on the computer, the virtual controller client comprising:
a button setting adjusting unit configured to receive first button setting information including a mapping relationship between key inputs to the application and associated virtual input messages, and to specify an arrangement and attributes of virtual buttons based on the received first button setting information;
a user virtual button interface configured to generate a first virtual button screen based on the first button setting information in which touch regions corresponding to the virtual buttons are visually displayed, and to display the first virtual button screen on a touch screen display device of the mobile terminal;
a touch event filter configured to generate touch input messages recognized as key input by the application, based on touch event objects that are generated from touch signals, of the touch regions corresponding to the virtual buttons, among touch signals input by the touch screen; and
a client message interfacing unit configured to convert the touch input message into a virtual input message in a form recognized by the virtual controller server, and to output, to the virtual controller server, the converted touch input message as the virtual input message,
wherein the mapping relationship dynamically redefines the virtual input message associated with a given key input of the key inputs based on an event in the application,
wherein the button setting adjusting unit is configured to receive second button setting information including the dynamically changed virtual message associated with the given key input, and
wherein, in response to an occurrence of the event in the application, the user virtual button interface is configured to generate and display, on the touch screen display device of the mobile terminal, a second virtual button screen based on the second button information.

2. The mobile terminal of claim 1 , wherein:
the user virtual button interface activates an acceleration sensor of the mobile terminal to enable a detection of movements of the mobile terminal, and
the virtual controller client further comprises:
an acceleration data filter configured to generate a movement input message that is mapped to a key input of the application, based on acceleration data that is generated based on an acceleration signal generated by the acceleration sensor; and
the client message interfacing unit operable to convert the touch input message or the movement input message into a virtual input message in a form recognized by the virtual controller server.

3. The mobile terminal of claim 1 , further comprising a computer-readable storage medium storing a program that is run by the virtual controller client.

4. A computer comprising:
a virtual controller server configured to remotely communicate with a virtual controller client running on a remote mobile terminal including a touch screen display device for remote key input on an application running on the computer, the virtual controller server comprising:
a button setting generating unit configured to generate and transmit, to the virtual controller client, first button setting information including a mapping relationship between key inputs to the application and associated virtual input messages;

a server message interfacing unit configured to transmit a setting message including the first button setting information to the virtual controller client, and to receive a virtual input message from the virtual controller client, the virtual input message being generated based on a touch on the touch screen display device of the mobile terminal; and
a key mapping unit configured to identify a key input value mapped to the received virtual input message based on the first button setting information,
wherein the mapping relationship dynamically redefines the virtual input message associated with a given key input based on an event in the application,
wherein the button setting generating unit is further configured to, in response to an occurrence of the event in the application, generate and transmit, to the virtual controller client, second button setting information including the dynamically changed virtual input message associated with the given key input, and
wherein the key mapping unit is further configured to identify the key input value mapped to the dynamically changed virtual input message based on the second button setting information.

5. The computer of claim 4 , wherein the mobile terminal further comprises an acceleration sensor configured to detect movements, and wherein the server message interfacing unit operable to receive a virtual input message generated based on a movement of the mobile terminal.

6. The computer of claim 4 , wherein the key mapping unit transfers a key input value to the application via a message transfer architecture of an operating system that runs the application on the computer.

7. The computer of claim 4 , wherein the key mapping unit transfers a key input value to the application via an input and output application programming interface (API) of an operating system that runs the application on the computer.

8. The computer of claim 4 , further comprising a computer-readable storage medium storing a program that is run by the virtual controller server.

9. A remote control system, comprising:
a computer including a virtual controller server configured to generate first button setting information including mapping relationship between key inputs to an application running on the computer and associated virtual input messages, and transfer the first button setting information to a virtual controller client, for extracting a key input from a virtual input message received from the virtual controller client, and for providing the key input to the application; and
a mobile terminal including a touch screen display device and the virtual controller client, the virtual controller client being configured to remotely communicate with the computer, for specifying an arrangement and attributes of virtual buttons based on the first button setting information received from the virtual controller server, for generating a first virtual button screen in which touch regions corresponding to the virtual buttons are visually displayed on the touch screen display device of the mobile terminal, for generating a touch input message recognized as a key input by the application, based on touch event objects generated based on touch signals for the touch regions corresponding to the virtual buttons, and for converting the touch input message into a virtual input message in a form recognized by the virtual controller server and output the virtual input message,
wherein the mapping relationship dynamically redefines the virtual input message associated with a given key input based on an event in the application,
wherein the virtual controller server is further configured to generate, in response to an occurrence of the event in the application, second button setting information including the dynamically changed virtual message associated with the given key input and transfer the second button setting information to the virtual controller client, and
wherein the virtual controller client is further configured to receive the second button setting information, and generate and display, on the touch screen display device of the mobile terminal, a second virtual button screen based on the second button setting information.

10. The remote control syste

Assignees

• Univ Sungkyunkwan Res & Bus
• Res & Business Found Sungkyunkwan Univeristy

Inventors

• Shin Dong Ryeol
• Nam Choon Sung

Classifications

• G06F3/048 Primary

  Interaction techniques based on graphical user interfaces [GUI] · CPC title

• A63F13/2145 Primary

  the surface being also a display device, e.g. touch screens · CPC title

• A63F13/22

  Setup operations, e.g. calibration, key configuration or button assignment · CPC title

• A63F13/06

  Human Necessities · mapped topic

• A63F13/426 Primary

  involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun · CPC title