Context-sensitive dynamic update of voice to text model in a voice-enabled electronic device

Patent metadata

Field	Value
Publication number	US-9966073-B2
Application number	US-201514723250-A
Country	US
Kind code	B2
Filing date	May 27, 2015
Priority date	May 27, 2015
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

A voice to text model used by a voice-enabled electronic device is dynamically and in a context-sensitive manner updated to facilitate recognition of entities that potentially may be spoken by a user in a voice input directed to the voice-enabled electronic device. The dynamic update to the voice to text model may be performed, for example, based upon processing of a first portion of a voice input, e.g., based upon detection of a particular type of voice action, and may be targeted to facilitate the recognition of entities that may occur in a later portion of the same voice input, e.g., entities that are particularly relevant to one or more parameters associated with a detected type of voice action.

First claim

What is claimed is:

1. A method, comprising:
receiving a voice input with a voice-enabled electronic device, the voice input including an original request that includes first and second portions, the second portion including a first context sensitive entity among a plurality of context sensitive entities that are associated with a context sensitive parameter and that potentially may be spoken in the voice input; and
in the voice-enabled electronic device, and responsive to receiving the first portion of the voice input:
performing local processing of the first portion of the voice input to dynamically build at least a portion of a voice action prior to completely receiving the voice input with the voice-enabled electronic device;
determining during the local processing whether the voice action is associated with the context sensitive parameter; and
in response to a determination that the voice action is associated with the context sensitive parameter and prior to performing local processing of the second portion of the voice input including the first context sensitive entity, initiating a dynamic update to a local voice to text model used by the voice-enabled electronic device prior to completing the voice action to facilitate recognition of the first context sensitive entity.

2. The method of claim 1 , wherein performing the local processing includes:
converting a digital audio signal of the voice input to text using a streaming voice to text module of the voice-enabled electronic device, wherein the streaming voice to text module dynamically generates a plurality of text tokens from the digital audio signal; and
dynamically building the portion of the voice action from at least a portion of the plurality of text tokens using a streaming semantic processor of the voice-enabled electronic device.

3. The method of claim 2 , wherein determining whether the voice action is associated with the context sensitive parameter is performed by the streaming semantic processor, and wherein initiating the dynamic update to the local voice to text model includes communicating data from the streaming semantic processor to the streaming voice to text module to initiate the dynamic update of the local voice to text model.

4. The method of claim 1 , wherein the local voice to text model comprises at least one decoding graph, and wherein initiating the dynamic update of the local voice to text model includes adding a decoding path to the at least one decoding graph corresponding to each of the plurality of context sensitive entities.

5. The method of claim 1 , further comprising, in response to a determination that the voice action is associated with the context sensitive parameter, prefetching from an online service voice to text model update data associated with the plurality of context sensitive entities, wherein initiating the dynamic update of the local voice to text model includes communicating the prefetched voice to text model update data to dynamically update the local voice to text model.

6. The method of claim 1 , wherein determining during the local processing whether the voice action is associated with a context sensitive parameter includes determining whether the voice action is a request to play a media item, wherein the context sensitive parameter includes a media item identifier for use in identifying the media item, and wherein the plurality of context sensitive entities identify a plurality of media items playable by the voice-enabled electronic device.

7. The method of claim 1 , wherein determining during the local processing whether the voice action is associated with a context sensitive parameter includes determining whether the voice action is a request to communicate with a contact, wherein the context sensitive parameter includes a contact identifier for use in initiating a communication with the contact, and wherein the plurality of context sensitive entities identify a plurality of contacts accessible by the voice-enabled electronic device.

8. The method of claim 1 , wherein the context sensitive parameter is a location-dependent parameter, and wherein the plurality of context sensitive entities identify a plurality of points of interest disposed in proximity to a predetermined location.

9. The method of claim 8 , wherein the predetermined location comprises a current location of the voice-enabled electronic device, the method further comprising, in response to a determination that the voice action is associated with the context sensitive parameter, communicating the current location to an online service and prefetching from the online service voice to text model update data associated with the plurality of context sensitive entities.

10. A method, comprising:
receiving a voice input including an original request including first and second portions with a voice-enabled electronic device, the voice input associated with a voice action having a context sensitive parameter, the first and second portions being different from one another and the second portion including a first context sensitive entity among a plurality of context sensitive entities that potentially may be spoken in the voice input;
performing voice to text conversion locally in the voice-enabled electronic device on the first portion of the voice input using a local voice to text model to generate text for the first portion of the voice input;
in response to determining that the voice action is associated with the context sensitive parameter, initiating a dynamic update to the local voice to text model after generating the text for the first portion of the voice input and prior to attempting to generate text for the second portion of the voice input to facilitate recognition of the plurality of context sensitive entities; and
performing voice to text conversion locally in the voice-enabled electronic device on the second portion of the voice input using the dynamically updated local voice to text model to generate text for the second portion of the voice input, the generated text including text for the first context sensitive entity.

11. The method of claim 10 , wherein performing the voice to text conversion includes converting a digital audio signal of the voice input to text using a streaming voice to text module of the voice-enabled electronic device, wherein the streaming voice to text module dynamically generates a plurality of text tokens from the digital audio signal, the method further comprising dynamically building at least a portion of the voice action prior to completely receiving the voice input with the voice-enabled electronic device from at least a portion of the plurality of text tokens using a streaming semantic processor of the voice-enabled electronic device.

12. The method of claim 11 , wherein initiating the dynamic update to the local voice to text model is performed by the streaming semantic processor.

13. An apparatus including memory and one or more processors operable to execute instructions stored in the memory, comprising instructions to:
receive a voice input with a voice-enabled electronic device, the voice input including an original request that includes first and second portions, the second portion including a first context sensitive entity among a plurality of context sensitive entities that are associated with a context sensitive parameter and that potentially may be spoken in the voice input; and
in the voice-enabled electronic device, and responsive to receiving the first portion of the voice input:
perform local processing of the first portion of the voice input to dynamically build at least a portion of a voice action prior to completely receiving t

Assignees

• Google Inc
• Google Llc

Inventors

• Gao Yuli
• Sung Sangsoo
• Murugesan Prathab

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/083

  Recognition networks (G10L15/142, G10L15/16 take precedence) · CPC title

• G10L2015/228

  of application context · CPC title

• G10L15/063

  Training · CPC title

• G10L15/26 Primary

  Speech to text systems (G10L15/08 takes precedence) · CPC title