Balance modifications of audio-based computer program output including a chatbot selected based on semantic processing of audio

Patent metadata

Field	Value
Publication number	US-10600409-B2
Application number	US-201715618854-A
Country	US
Kind code	B2
Filing date	Jun 9, 2017
Priority date	Jun 9, 2017
Publication date	Mar 24, 2020
Grant date	Mar 24, 2020

Abstract

Modifying computer program output in a voice or non-text input activated environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to select a computer program, such as a chatbot, to invoke based on semantic processing of the audio signal. The computer program can identify a dialog data structure. The system can modify the identified dialog data structure to include a content item. The system can provide the modified dialog data structure to a computing device for presentation.

First claim

What is claimed is:

1. A system to balance data requests for modification of computer program output, comprising:
a data processing system having one or more processors and memory to:
receive, from a computing device, a first digital file corresponding to a first acoustic signal from a user with first voice content detected by a microphone of the computing device, the first acoustic signal converted to the first digital file by an analog to digital converter of the computing device;
select, from a data repository identifying a plurality of computer programs comprising chatbots, responsive to receipt of the first digital file corresponding to the first voice content from the user detected by the microphone of the computing device and based on semantic processing of the first voice content by the data processing system prior to chatbot execution, a computer program comprising a chatbot from the plurality of computer programs comprising chatbots for execution;
identify, via the chatbot based on the first voice content of the first digital file, a first dialog data structure comprising a first placeholder field;
select, via a content selection process responsive to identification of the first placeholder field in the first dialog data structure, a content item for insertion into the first placeholder field of the first dialog data structure, the content item in a parameterized format configured for a parametrically driven text to speech technique;
provide, to the chatbot, the content item in the parameterized format selected via the content selection process to cause the computing device to perform the parametrically driven text to speech technique to generate a second acoustic signal corresponding to the first dialog data structure modified with the content item;
generate an index value based on a first identifier of the chatbot, a second identifier for the first dialog data structure, and a third identifier for the computing device;
associate, in the memory, the content item with the index value;
receive a second digital file corresponding to a third acoustic signal carrying second voice content detected by the microphone on the computing device;
select, responsive to the second voice content of the second digital file, the computer program comprising the chatbot;
identify, via the chatbot based on the second voice content of the second digital file, a second dialog data structure comprising a second placeholder field;
select, responsive to identification of the second placeholder field and based on the first identifier of the chatbot, the third identifier of the computing device, and a fourth identifier of the second dialog data structure, the content item associated with the index value; and
provide, to the chatbot, the content item associated with the index value to cause the computing device to perform the parametrically driven text to speech technique to generate a fourth acoustic signal corresponding to the second dialog data structure modified with the content item.

2. The system of claim 1 , comprising the data processing system to:
establish a session responsive to the first digital file;
determine a break in the session subsequent to provision of the first dialog data structure; and
resume the session based on the second digital file.

3. The system of claim 1 , comprising the data processing system to:
establish a session responsive to the first digital file;
determine, based on at least one of a temporal threshold or a location threshold, a break in the session subsequent to provision of the first dialog data structure; and
resume the session based on the second digital file.

4. The system of claim 1 , comprising the data processing system to:
establish a session responsive to the first digital file;
determine, based on a natural language processing technique, a break in the session subsequent to provision of the first dialog data structure; and
resume the session based on application of the natural language processing technique to the second digital file.

5. The system of claim 1 , comprising the data processing system to:
generate the index value based on application of a hash function to the first identifier, the second identifier, and the third identifier; and
store the index value in a hash table in memory.

6. The system of claim 1 , comprising the data processing system to:
generate the index value based on application of a hash function to the first identifier, the second identifier, and the third identifier;
associate the index value with the content item in a hash table in memory;
identify the index value based on application of the hash function to the first identifier, the third identifier, and the fourth identifier, wherein the fourth identifier corresponds to the second identifier; and
retrieve, from the hash table, the content item associated with the index value for provision with the second dialog data structure.

7. The system of claim 1 , comprising the data processing system to:
receive a third digital file corresponding to a fifth acoustic signal carrying third voice content detected by the microphone on the computing device;
select, responsive to the third voice content of the third digital file, the computer program comprising the chatbot;
identify, via the chatbot based on the third voice content of the third digital file, a third dialog data structure comprising a third placeholder field;
generate a second index value based on the first identifier, the third identifier, and a fifth identifier of the third dialog data structure;
determine, based on a comparison of the index value with the second index value, not to reuse the content item; and
select, responsive to identification of the third placeholder field and based on the first identifier of the chatbot, the third identifier of the computing device, and the fifth identifier of the third dialog data structure, a second content item to provision to the computing device to cause the computing device to perform the parametrically driven text to speech technique to generate a sixth acoustic signal corresponding to the third dialog data structure modified with the second content item.

8. The system of claim 1 , comprising the data processing system to:
determine, based on the third identifier of the computing device and via a lookup in the data repository, that the computing device is authorized to access the computer program comprising the chatbot; and
select the computer program comprising the chatbot responsive to the determination that the computing device is authorized to access the chatbot.

9. The system of claim 1 , comprising the chatbot to:
use a natural language processing technique to identify the first dialog data structure responsive to the first digital file.

10. The system of claim 1 , comprising the computing device to:
play the content item with an acoustic fingerprint corresponding to the chatbot.

11. A method of balancing data requests for modification of computer program output, comprising:
receiving, by a data processing system from a computing device, a first digital file corresponding to a first acoustic signal from a user carrying first voice content detected by a microphone of the computing device, the first acoustic signal converted to the first digital file by an analog to digital converter of the computing device;
selecting, by the data processing system responsive to receipt of the first digital file corresponding to the first voice content from the user detected by the microphone of the computing device and based on semantic processing of the first voice content by the data processing system prior to chatbot execution, a computer prog

Assignees

• Google Inc
• Google Llc

Inventors

• Eidem Laura
• Jacobson Alex

Classifications

• G06F16/61

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/685

  using automatically derived transcript of audio data, e.g. lyrics (speech recognition G10L15/00) · CPC title

• G06F16/433

  using audio data · CPC title

• G10L13/04 Primary

  Details of speech synthesis systems, e.g. synthesiser structure or memory management · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title