Systems and methods for processing positional and sound input of user input to a touch panel

Patent metadata

Field	Value
Publication number	US-9498708-B2
Application number	US-33386708-A
Country	US
Kind code	B2
Filing date	Dec 12, 2008
Priority date	Dec 25, 2007
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

A game apparatus detects an input position, on an input plane, at which an input operation has been performed. The game apparatus also detects an operation sound that occurs due to the input operation performed on the input plane. The game apparatus evaluates the detected operation sound, and obtains a result of the evaluation (e.g., volume of the operation sound). The game apparatus performs game processing by using the detected input position and the result of the evaluation. This enables the game apparatus to perform more complex processing by using a touch operation as input information, and allows a player to provide more complex instructions based on the touch operation.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium storing computer readable instructions to be executed by a computing system that includes an input device having a touch screen display that is configured to detect a touch operation that is performed by a user, and a microphone, the stored instructions comprising instructions that are configured to:
output an image to the touch screen display;
obtain, from the input device, an input position which corresponds to an input operation performed by a user at a touched position on the touch screen display while the image is displayed on the touch screen display;
obtain, from the microphone, an operation sound based on the performed input operation; and
perform a game process based on the obtained input position and the obtained operation sound,
wherein the input position is a coordinate input position,
wherein the input position is obtained independently of the operation sound.

2. The medium of claim 1 , wherein the instructions are further configured to calculate, by performing a calculation process that is based on the operation sound, feature information indicating a feature of the operation sound,
wherein the game process is further based on the feature information, and
the calculation process determines a detail in accordance with the obtained input position.

3. The medium of claim 2 , wherein the instructions are further configured to:
determine a characteristic of a filter based on the input position; and
perform a filtering process on the operation sound, by using the determined characteristic of the filter, to obtain filtered operation sound,
wherein the calculation of the feature information from the operation sound is further based on the filtered operation sound.

4. The medium of claim 3 , wherein the instructions are further configured to determine a frequency passband that is to be used for the filter based on the input position.

5. The medium of claim 2 , wherein the instructions are further configured to:
determine an amplification rate for amplifying a signal of the operation sound based on the input position;
amplify the signal of the operation sound based on the determined amplification rate to obtain amplified operation sound data; and
calculate a volume of the operation sound as the feature information based on the amplified operation sound data.

6. The medium of claim 2 , wherein the instructions are further configured to:
instruct a user to perform a second input operation on the touch screen display;
obtain, from the microphone, a second operation sound occurring due to the second input operation that is performed at a position within the predetermined area of the touch screen display;
determine, based on the second operation sound, the detail; and
generate and store association data that associates the predetermined area with the determined detail,
wherein the predetermined calculation process determines the detail based on the obtained input position being within the predetermined area of the stored association data.

7. The medium of claim 2 , wherein the instructions are further configured to:
instruct a user to perform a second input operation on the touch screen display;
obtain, from the microphone, a second operation sound occurring due the second input operation that is performed on a second position within a predetermined area of the touch screen display;
generate and store association data that associates the predetermined area with the second operation sound;
select the second operation sound from the association data when the input position is within the predetermined area associated with the second operation sound; and
compare the second operation sound to the operation sound.

8. The medium of claim 1 , wherein the instructions are further configured to:
determine whether or not the obtained operation sound satisfies a predetermined standard, the predetermined standard including using different reference values depending on the obtained input position; and
perform a game process using the obtained input position when the operation sound satisfies the predetermined standard.

9. The medium of claim 8 , wherein the instructions are further configured to:
instruct a user to perform a second input operation on the touch screen display;
obtain, from the microphone, a second operation sound occurring due to the second input operation that is performed, in response to a provided instruction, at a second position within a predetermined area of the touch screen display;
set, based on the obtained second operation sound, a reference value used for when the input operation is performed on a position within the predetermined area; and
generate and store association data which associates the predetermined area with the set reference value,
wherein the game process uses the reference value which is associated with the predetermined area stored in the association data.

10. The medium of claim 1 , wherein the instructions are further configured to:
calculate, by performing a predetermined calculation process on the operation sound, feature information indicating a feature of the operation sound; and
correct the feature information in accordance with the input position.

11. The medium of claim 10 , wherein the instructions are further configured to calculate, as the feature information, volume of the operation sound.

12. The medium of claim 1 , wherein the game process includes causing a controlled object to perform a predetermined action based on the input position and the operation sound.

13. The medium of claim 12 , wherein
the image includes a first object,
wherein the game process includes:
determining, when the input position and a display position of the first object are in a predetermined relationship, whether or not to cause a second object, which is the same as or different from the first object, to perform the predetermined action; and
causing, when the second object is to perform the predetermined action, the second object to perform an action that corresponds to the operation sound.

14. The medium of claim 12 , wherein
the image includes a first object,
wherein the game process includes:
specifying, when the input position and a display position of the first object are in a predetermined relationship, a second object which is the same as or different from the first object; and
causing the second object to perform an action corresponding to the operation sound.

15. The medium of claim 12 , wherein
the image includes the controlled object,
wherein the game process includes:
determining, based on the input position, a value of a first game parameter relating to the predetermined action of the controlled object; and
determining, based on the operation sound, a value of a second game parameter that is different from the first game parameter and which relates to the predetermined action of the controlled object.

16. The medium of claim 13 , wherein the instructions are further configured to determine, based on the volume of the operation sound, an amount of the action that corresponds to the operation sound of the second object.

17. The medium of claim 13 , wherein the instructions are further configured to determine, based on the operation sound, a type of the action of the second object.

18. The medium of claim 1 , wherein the instructions are further configured to:
specify, based on the input position, a position in a game space; and
determine, based on the operation sound, a detail o

Assignees

• Masuda Takumi
• Kanata Ryosuke
• Nakano Yoshinobu
• Nishimura Masahiko
• Nintendo Co Ltd

Inventors

• Masuda Takumi
• Kanata Ryosuke
• Nakano Yoshinobu
• Nishimura Masahiko

Classifications

• A63F2300/1006

  having additional degrees of freedom · CPC title

• A63F2300/6045

  for mapping control signals received from the input arrangement into game commands · CPC title

• A63F13/2145

  the surface being also a display device, e.g. touch screens · CPC title

• A63F2300/204

  the platform being a handheld device · CPC title

• A63F2300/1068

  being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad · CPC title