Speech-based audio indicators

Patent metadata

Field	Value
Publication number	US-10409552-B1
Application number	US-201615269755-A
Country	US
Kind code	B1
Filing date	Sep 19, 2016
Priority date	Sep 19, 2016
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

Systems and methods for displaying an audio indicator including a main portion having a width proportional to a volume of a particular phoneme of an utterance is described herein. In some embodiments, audio data representing an utterance may be received at a speech-processing system from a user device. The speech-processing system may determine a maximum volume amplitude for the utterance, and using the maximum volume amplitude, may determine a normalized amplitude value between 0 and 1 associated with a volume that phoneme's of an utterance are spoken. The speech-processing system may then map the normalized amplitude value(s) to widths for a main portion of an audio indicator, where larger normalized amplitude values may correspond to smaller main portion widths. Display data representing a graphical user interface may then be generated and sent to the user device, where the graphical user interface includes the audio indicator having its main portion of a particular width representative of a volume of each phoneme from the utterance.

First claim

What is claimed is:

1. A method, comprising:
receiving, at an electronic device, first audio data representing a first portion of an utterance including a first phoneme, the first portion being of a first temporal duration;
determining a first volume amplitude associated with the first temporal duration using the first audio data;
determining, using the first volume amplitude, a first normalized amplitude value for the first temporal duration relative to a preconfigured maximum volume amplitude value and a preconfigured minimum volume amplitude value;
determining a first horizontal width of an audio indicator based on the first normalized amplitude value;
generating first display data representing a first graphical user interface (“GUI”) including the audio indicator, wherein the audio indicator has a plurality of portions with at least one portion based on the amplitude of the first audio data and a first height that is independent of the first volume amplitude; and
displaying the audio indicator using a display screen of the electronic device to visually indicate a first apparent loudness of the first phoneme.

2. The method of claim 1 , further comprising:
receiving, at the electronic device, second audio data representing a second portion of the utterance including a second phoneme, the second portion being of a second temporal duration;
determining a second volume amplitude associated with the second temporal duration using the second audio data;
determining, using the second volume amplitude, a second normalized amplitude value for the second temporal duration relative to the preconfigured maximum volume amplitude value and the preconfigured minimum volume amplitude value;
determining a second horizontal width of the audio indicator based on the second normalized amplitude value, the second horizontal width being one of: greater than the first horizontal width or less than the first horizontal width;
generating second display data representing a second GUI including the audio indicator having the second horizontal width and the first height; and
displaying the audio indicator using the display screen visually indicate a second apparent loudness of the second phoneme.

3. The method of claim 1 , further comprising:
determining, prior to generating the first display data, that the first portion originated from a first physical location relative to the electronic device;
determining that the audio indicator is to be centered about a first origin point such that the audio indicator visually indicates an apparent positioning of where the utterance emanated from relative to the electronic device, the first origin point visually indicating a relative horizontal location that the utterance emanated from relative to the electronic device.

4. The method of claim 1 , further comprising:
receiving, prior to generating the first display data, first image data representing a first image captured by a first camera of the electronic device;
determining a first location of an individual to the display screen based on the first image data;
determining that the audio indicator is the first height based on the first image data, where the first height represents a distance from the utterance to the electronic device such that the audio indicator has a larger height for utterances emanating one of closer to or farther away from the display screen.

5. A method, comprising:
receiving first audio data representing a first utterance;
determining a first sound value associated with the first utterance;
determining, based at least in part on the first sound value, a first indicator value associated with the first utterance, the first indicator value satisfying a first sound threshold;
generating first display data representing a first graphical beam, the first graphical beam having a first portion, a second portion, and a third portion being a first size based, at least in part, on the first indicator value, and the first portion being visually distinguishable from the second portion and the third portion; and
presenting the first display data.

6. The method of claim 5 , further comprising:
determining a second sound value associated with the first utterance;
determining, based at least in part on the second sound value, a second indicator value associated with the first utterance, the second indicator value satisfying the first sound threshold;
generating second display data representing the first graphical beam having the first portion, the second portion, and the third portion being of a second size based, at least in part, on the second indicator value; and
presenting the second display data.

7. The method of claim 6 , wherein determining the second indicator value further comprises:
determining that the first sound value corresponds to a first amplitude value;
determining that the second sound value corresponds to a second amplitude value, the second amplitude value being greater than the first amplitude value; and
determining that the second indicator value is greater than the first indicator value.

8. The method of claim 5 , further comprising:
determining, prior to receiving the first audio data, that the first audio data originated from a first location relative to a user device;
determining that the first location is associated with a first side of the user device; and
determining that the first portion is centered about a first origin point based, at least in part, on the first location.

9. The method of claim 8 , further comprising:
receiving second audio data representing a second utterance;
determining that the second audio data originating from a second location;
determining a second sound value associated with the second utterance;
determining, based at least in part on the second sound value, a second indicator value associated with the second utterance, the second indicator value satisfying the first sound threshold;
generating second display data representing a second graphical beam having a fourth portion and a fifth portion being of a second sized based, at least in part, on the second indicator value, the fourth portion being centered about a second origin point based, at least in part, on the second location; and
presenting the second display data at a substantially same time as the first display data such that the first portion is orientated at a first position on the first graphical beam corresponding to the first origin point, and the fourth portion is oriented at a second position on the second graphical beam corresponding to the second origin point.

10. The method of claim 5 , further comprising:
sending the audio data to an electronic device to determine a speech endpoint of the first utterance;
receiving an instruction from the electronic device indicating that the speech endpoint has been identified; and
presenting second display data representing a speech endpoint graphic.

11. The method of claim 5 , further comprising:
receiving, prior to generating the first display data, first image data representing a first image;
determining that the first image includes an individual;
determining a first distance between the individual and a user device that displays the first graphical beam; and
determining a second indicator value associated with the first distance.

12. The method of claim 11 , further comprising:
causing the first portion to be of the first size based on the first indicator value and the second indicator value, the first indicator value corresponding to a width of the first portion, and the second indicator value corresponding to a height of the first portion.

13. A syst

Assignees

• Amazon Tech Inc

Inventors

• Jara David Adrian
• Gray Timothy Thomas
• Lee Kwan Ting
• Park Jae Pum
• Hone Michael
• Hinkson Grant
• Mains Richard Leigh
• Bhagat Shilpan

Classifications

• G10L2015/025

  Phonemes, fenemes or fenones being the recognition units · CPC title

• G10L15/05

  Word boundary detection · CPC title

• G10L15/02

  Feature extraction for speech recognition; Selection of recognition unit · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L2015/225

  Feedback of the input speech · CPC title