Information processing system, information processing apparatus, storage medium having stored therein information processing program, and information processing method

Patent metadata

Field	Value
Publication number	US-10796540-B2
Application number	US-201514686094-A
Country	US
Kind code	B2
Filing date	Apr 14, 2015
Priority date	May 30, 2014
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

A sound source localization position indicating a position where a sound source is localized is set, and a sound signal for localizing the sound source at the set sound source localization position is generated and output. Then, a vibration signal for, in conjunction with the sound source localization position, vibrating a vibrator for imparting a vibration to a user is generated and output.

First claim

What is claimed is:

1. A system comprising one or more speakers and an information processing apparatus in communication with the speakers, wherein
the information processing apparatus comprises:
a hand-holdable housing;
a display disposed on the front surface of the housing;
first and second spaced apart vibrators disposed in the housing; and
a processor configured to:
control the display to display, based on inputs to the information processing apparatus, an image of a virtual space, the image of the virtual space including an object;
generate, based on a sound source in the virtual space, sound signals for use in outputting, via the speakers, sound for user perception as being localized at a sound localization position in real space outside the housing of the information processing apparatus, the sound localization position being based on a position at which the object is displayed on the display; and
generate, based on a vibration source in the virtual space, vibration signals for controlling the first and second vibrators to generate vibration of the housing that is localized to remain on a straight line in real space with a user position and the sound localization position as the object changes position on the display, thereby causing user perception of the generated vibration as being localized at a vibration localization position which is the same as the sound localization position to continue as the sound localization position shifts based on the changing position of the object.

2. The information processing system according to claim 1 , wherein
the processor is configured to generate the vibration signals for changing an intensity of the vibration generated by the first and second vibrators in accordance with the sound localization position.

3. The information processing system according to claim 1 , wherein
the processor is configured to generate vibration signals for changing proportions of intensities of the vibration generated by the first and second vibrators.

4. The information processing system according to claim 1 , wherein
the processor is configured to generate the sound signals based on a function for calculating sound from the sound source to each of a left ear and a right ear.

5. The information processing system according to claim 4 , wherein
the function is based on a sound pressure level characteristic corresponding to a frequency.

6. The information processing system according to claim 4 , wherein
the function is based on a sound pressure level characteristic corresponding to a direction of the sound source.

7. The information processing system according to claim 4 , wherein
the function is a head-related transfer function.

8. The information processing system according to claim 1 , wherein
the processor is configured to generate the sound signals using data in which sound from the sound source to each of a left ear and a right ear is recorded.

9. The information processing system according to claim 1 , wherein
the processor is configured to generate the vibration signals for changing an intensity of the vibration generated by the first and second vibrators in accordance with a distance from the sound localization position.

10. The information processing system according to claim 1 , wherein
the one or more speakers are included in headphones.

11. The system according to claim 1 , wherein the one or more speakers is in wireless communication with the information processing apparatus.

12. The system according to claim 1 , wherein the one or more speakers is in wired communication with the information processing apparatus.

13. The system according to claim 1 , further comprising:
a sensor configured to sense aspects of orientation and/or motion of the information processing apparatus.

14. An information processing apparatus comprising:
a hand-holdable housing;
a display disposed on a front surface of the housing;
first and second spaced apart vibrators disposed in the housing; and
a processor configured to:
control the display to display, based on inputs to the information processing apparatus, an image of a virtual space, the image of the virtual space including an object;
generate, based on a sound source in the virtual space, sound signals for use in outputting, via speakers in communication with the information processing apparatus, sound for user perception as being localized at a sound source localization position in real space outside the housing of the information processing apparatus, the sound localization position being based on a position at which the object is displayed on the display; and
generate, based on a vibration source in the virtual space, vibration signals for controlling the first and second vibrators to generate vibration of the housing that is localized to remain on a straight line in real space with a user position and the sound localization position as the object changes position on the display, thereby causing user perception of the generated vibration as being localized at a vibration localization position which is the same as the sound localization position to continue as the sound localization position shifts based on the changing position of the object.

15. A non-transitory computer-readable storage medium having stored therein an information processing program for execution by a processor included in an information processing apparatus comprising a hand-holdable housing; a display disposed on a front surface of the housing; and first and second spaced apart vibrators disposed in the housing,
the information processing program, when executed, causing the processor to control the information processing apparatus to operations comprising:
control the display to display, based on inputs to the information processing apparatus, an image of a virtual space, the image of the virtual space including an object;
generating, based on a sound source in the virtual space, sound signals for use in outputting, via speakers in communication with the information processing apparatus, sound for user perception as being localized at a sound localization position in real space outside the housing of the information processing apparatus, the sound localization position being based on a position at which the object is displayed on the display; and
generating, based on a vibration source in the virtual space, vibration signals for controlling the first and second vibrators to generate vibration of the housing that is localized to remain on a straight line in real space with a user position and the sound localization position as the object changes position on the display, thereby causing user perception of the generated vibration as being localized at a vibration localization position which is the same as the sound localization position to continue as the sound localization position shifts based on the changing position of the object.

16. An information processing method performed by a processor or cooperation of a plurality of processors included in an information processing apparatus comprising a hand-holdable housing, a display disposed on a front surface of the housing, and first and second spaced-apart vibrators disposed in the housing, the method comprising:
displaying, on the display, based on inputs to the information processing apparatus, an image of a virtual space, the image of the virtual space including an object;
generating, based on a sound source in the virtual space, sound signals for use in outputting, via speakers in communication with the information processing apparatus, sound for user perception as

Assignees

• Nintendo Co Ltd

Inventors

• Obana Kazutoshi
• Aoki Takafumi
• Kawai Kochi
• Miyatake Junichiro
• Yamashita Kei
• Masubuchi Shoji
• Inami Masahiko
• Minamizawa Kouta

Classifications

• A63F13/54 Primary

  involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall · CPC title

• A63F2300/204

  the platform being a handheld device · CPC title

• A63F13/28

  responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light · CPC title

• G08B6/00 Primary

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title