Visual feedback for speech recognition system

Patent metadata

Field	Value
Publication number	US-9721587-B2
Application number	US-201313749392-A
Country	US
Kind code	B2
Filing date	Jan 24, 2013
Priority date	Jan 24, 2013
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

Embodiments are disclosed that relate to providing visual feedback in a speech recognition system. For example, one disclosed embodiment provides a method including displaying a graphical feedback indicator having a variable appearance dependent upon a state of the speech recognition system. The method further comprises receiving a speech input, modifying an appearance of the graphical feedback indicator in a first manner if the speech input is heard and understood by the system, and modifying the appearance of the graphical feedback indicator in a different manner than the first manner if the speech input is heard and not understood.

First claim

The invention claimed is:

1. On a computing device, a method of providing user feedback for a speech recognition system, the method comprising:
displaying a graphical feedback indicator with an appearance that indicates the speech recognition system is in a passive listening mode;
receiving a triggering input;
in response to the triggering input, modifying an appearance of the graphical feedback indicator to have an appearance that indicates the speech recognition system is in an active listening mode;
while in the active listening mode, receiving a speech input;
when the speech recognition system understands the speech input, modifying the appearance of the graphical feedback indicator to have an appearance that indicates that the speech input is understood by the system, determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user by adjusting a location of a volume indicator in a direction corresponding to the location of the user, the volume indicator comprising a ring-shaped portion of the indicator having a bar that represents a volume, wherein adjusting a location indicator comprises moving the bar to a side corresponding to the location of the user providing the speech input; and
when the speech recognition system does not understand the speech input, then modifying the appearance of the graphical feedback indicator in a different manner to have an appearance that indicates the speech input is not understood by the system.

2. The method of claim 1 , further comprising determining the volume of the speech input and modifying a length of the volume indicator to indicate the volume.

3. The method of claim 1 , further comprising modifying the appearance of the graphical feedback indicator during a continuous speech recognition mode to display one or more words of the speech input in real-time as each word is recognized.

4. The method of claim 1 , further comprising identifying the user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate an identity of the user.

5. The method of claim 1 , wherein modifying the appearance of the graphical feedback indicator in the different manner comprises displaying a prompt for the user to provide additional user input.

6. The method of claim 1 , further comprising modifying the appearance of the graphical feedback indicator from the passive listening mode to a different, active listening mode by maintaining a shape of the graphical feedback indicator while modifying the appearance.

7. A method of providing feedback for a speech recognition system of a computing device, the method comprising:
displaying a graphical feedback indicator with an appearance indicating that the speech recognition system is in a passive listening mode, the graphical feedback indicator having a variable appearance depending upon a state of the speech recognition system;
receiving a triggering input;
in response to the triggering input, modifying the appearance of the graphical feedback indicator to have an appearance that indicates the speech recognition system is in an active listening mode;
while in the active listening mode, receiving a speech input;
when the speech input is heard, modifying the appearance of the graphical feedback indicator to indicate that the speech input is heard;
when the speech input is understood, modifying the appearance of the graphical feedback indicator to indicate that the speech input is understood, determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user by adjusting a location of a volume indicator in a direction corresponding to the location of the user, the volume indicator comprising a ring-shaped portion of the indicator having a bar that represents a volume, wherein adjusting a location indicator comprises moving the bar to a side corresponding to the location of the user providing the speech input; and
when the speech input is not understood, modifying the appearance of the graphical feedback indicator to indicate that the speech is not understood.

8. The method of claim 7 , wherein the appearance that indicates that the speech recognition system is in the active listening mode comprises an appearance that indicates the speech recognition system is in the active listening mode in a local context.

9. The method of claim 7 , wherein the appearance that indicates that the speech recognition system is in the active listening mode comprises an appearance that indicates that the speech recognition system is in the active listening mode in a global context.

10. The method of claim 7 , further comprising modifying the appearance of the graphical feedback indicator during a continuous speech recognition mode to indicate one or more words of the speech input as the speech input is recognized.

11. The method of claim 7 , further comprising modifying the appearance of the graphical feedback indicator to identify one or more words corresponding to a canonical form of a speech command corresponding to the speech input.

12. The method of claim 7 , further comprising modifying the appearance of the graphical feedback indicator to indicate an uncertainty in a recognition of the speech input.

13. The method of claim 12 , wherein indicating the uncertainty in the recognition of the speech input includes displaying two or more possible speech recognition results for the speech input.

14. A computing system for performing speech recognition and providing feedback regarding the speech recognition, the computing system comprising:
a logic machine; and
a storage machine comprising instructions executable by the logic machine to:
output to a display device a graphical feedback indicator with an appearance indicating that a speech recognition system is in a passive listening mode, the graphical feedback indicator having a variable appearance depending upon a state of the speech recognition system;
receiving a triggering input;
in response to the triggering input, modifying an appearance of the graphical feedback indicator to have an appearance that indicates that the speech recognition system is in an active listening mode;
receive, from one or more microphones, a speech input;
when the speech input is heard, modify the appearance of the graphical feedback indicator to indicate that the speech input is heard; and
when the speech input is understood, modify the appearance of the graphical feedback indicator to indicate that the speech input is understood, determining a location of a user providing the speech input, and modifying the appearance of the graphical feedback indicator to indicate the location of the user by adjusting a location of a volume indicator in a direction corresponding to the location of the user, the volume indicator comprising a ring-shaped portion of the indicator having a bar that represents a volume, wherein adjusting a location indicator comprises moving the bar to a side corresponding to the location of the user providing the speech input.

15. The computing system of claim 14 , the instructions being further executable to modify the appearance of the graphical feedback indicator to indicate whether the speech input is to be applied in a global operating system context or a local application context.

16. The computing system of claim 14 , the instructions being further executable to determine an identity of the user, and to modify the appearance of the graphical feedback in

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Klein Christian
• Niman Meg

Classifications

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G10L21/10 Primary

  Transforming into visible information · CPC title

• G10L2015/225

  Feedback of the input speech · CPC title