Storage medium storing information processing program, information processing device, information processing method, and information processing system

Patent metadata

Field	Value
Publication number	US-10096257-B2
Application number	US-201213593779-A
Country	US
Kind code	B2
Filing date	Aug 24, 2012
Priority date	Apr 5, 2012
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

A new information processing device for presenting information and allowing a user to input sound in relation to the presented information is provided. An information processing device 10 includes: a display section 104 that displays a sentence that is a reading-aloud target; a sound input section 102 to which sound is input; a time measuring section 105 that measures a sound input time in which the sound is input to the sound input section 102 ; and a test execution section 103 that determines whether or not the reading-aloud target is read aloud based on the sound input time and a silent time after the sound is input to the sound input section.

First claim

What is claimed is:

1. A non-transitory storage medium having stored therein an information processing program, the information processing program when executed causing an information processing device to perform:
displaying, on a touch screen including a display panel and a touch sensor, a sentence including a target word that is displayed with a display mode different from a display mode of other words in the sentence;
receiving sound data corresponding to a sound captured by a sound input device configured to capture sound while the sentence is displayed; and
processing the received sound data to determine a silent time in which sound in the received sound data is absent or below a sound threshold and a sound input time in which sound is present in the received sound data;
determining whether or not the sentence is read aloud based on the sound input time and the silent time, wherein the sentence is determined to be read aloud when a continued amount of the silent time reaches a silent threshold value after an accumulated amount of the sound input time separated by one or more silent times reaches read-aloud completion time determined based on the number of words in the displayed sentence;
upon determining that the sentence is read aloud, controlling the display panel to stop displaying the sentence and display an answer input screen requesting a user to manually input the target word;
receiving, from the touch sensor, touch input data corresponding to input made to the answer input screen;
performing character recognition processing to determine whether the received touch input data represents characters corresponding to characters in the target word; and
upon determining that the received touch input data represents characters corresponding to characters in the target word, controlling the display panel to display another sentence including another target word.

2. The non-transitory storage medium according to claim 1 , wherein the information processing program further causes the information processing device to perform:
accumulating the sound input time while the sound is present in the received sound data regardless of whether or not there is a silent time after the sound in the received sound data starts.

3. The non-transitory storage medium according to claim 1 , wherein the read-aloud completion time is determined each time a new sentence is displayed based on the number of words in the displayed sentence.

4. The non-transitory storage medium according to claim 3 , wherein the information processing program causes the information processing device to perform:
displaying an object for causing a user to instruct termination of displaying the sentence while continuing the display of the sentence when the determination is made that the sentence is read aloud.

5. The non-transitory storage medium according to claim 4 , wherein when the silent time, during which there is no instruction for the object, reaches a third threshold value after the object is displayed, prompting an instruction of the user for the object.

6. The non-transitory storage medium according to claim 1 , wherein the determination that the sentence is read aloud is made regardless of the silent time when the sound input time reaches a third threshold value.

7. The non-transitory storage medium according to claim 1 , wherein the target word in the sentence is displayed with a mark.

8. An information processing method, implemented at least in part by a computer processor coupled to a touch screen including a display panel and a touch sensor and a sound input device configured to capture sound, the method comprising:
displaying, on the display panel, a sentence including a target word that is displayed with a display mode different from a display mode of other words in the sentence;
receiving sound data corresponding to a sound captured by the sound input device while the sentence is displayed; and
processing the received sound data to determine a silent time in which sound in the received sound data is absent or below a sound threshold and a sound input time in which sound is present in the received sound data;
determining whether or not the sentence is read aloud based on the sound input time and the silent time, wherein the sentence is determined to be read aloud when a continued amount of the silent time reaches a silent threshold value after an accumulated amount of the sound input time separated by one or more silent times reaches read-aloud completion time determined based on the number of words in the displayed sentence;
upon determining that the sentence is read aloud, controlling the display panel to stop displaying the sentence and display an answer input screen requesting a user to manually input the target word;
receiving, from the touch sensor, touch input data corresponding to input made to the answer input screen;
performing character recognition processing to determine whether the received touch input data represents characters corresponding to characters in the target word; and
upon determining that the received touch input data represents characters corresponding to characters in the target word, controlling the display panel to display another sentence including another target word.

9. The information processing method according to claim 8 ,
wherein the sound input time is accumulated while the sound is present in the received sound signal regardless of whether or not there is a silent time after the sound in the received sound signal starts.

10. The information processing method according to claim 8 , wherein the read-aloud completion time is determined each time a new sentence is displayed based on the number of words in the displayed sentence.

11. The information processing method according to claim 10 , further comprising
displaying an object for causing a user to instruct termination of displaying the sentence.

12. The information processing method according to claim 11 , further comprising
performing processing for prompting an instruction of the user for the object when the silent time, during which there is no instruction for the object, reaches a third threshold value after displaying the object.

13. The information processing method according to claim 8 , wherein the sentence is determined to be read aloud regardless of the silent time when the sound input time reaches a third threshold value.

14. The information processing method according to claim 8 ,
wherein target words in the sentence is displayed with a mark.

15. The information processing method to claim 8 , wherein the accumulated amount of the sound input time includes one or more silent times that do not reach the silent threshold value.

16. The information processing method to claim 8 , wherein the plurality of words in the sentence are simultaneously displayed, and immediately after determining that the sentence is read aloud, remove the sentence from the display panel.

17. The information processing method according to claim 8 , wherein the read-aloud completion time is determined by multiplying the number of words in the sentence by a factor representing a difficulty level of the sentence.

18. The non-transitory storage medium according to claim 1 wherein the information processing device is a game device, and the sound input device is disposed inside the housing of the game device.

19. The information processing method according to claim 8 , wherein the method further comprises, after it is determined that the sentence is read aloud, update the rea

Assignees

• Ito Yuichiro
• Jin Toshikazu
• Kawashima Ryuta
• Nintendo Co Ltd
• Univ Tohoku

Inventors

• Ito Yuichiro
• Jin Toshikazu
• Kawashima Ryuta

Classifications

• G09B5/04 Primary

  with audible presentation of the material to be studied (sound recording or reproducing G11B) · CPC title

• A63F13/80

  Special adaptations for executing a specific game genre or game mode · CPC title

• A63F13/424

  involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition · CPC title

• G10L25/51

  for comparison or discrimination · CPC title

• G09B19/06

  Foreign languages (with audible presentation of material to be studied G09B5/04) · CPC title