Modification of audio-based computer program output

Patent metadata

Field	Value
Publication number	US-11582169-B2
Application number	US-202017094361-A
Country	US
Kind code	B2
Filing date	Nov 10, 2020
Priority date	Jun 9, 2017
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

Modifying computer program output in a voice or non-text input activated environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to identify a computer program to invoke. The computer program can identify a dialog data structure. The system can modify the identified dialog data structure to include a content item. The system can provide the modified dialog data structure to a computing device for presentation.

First claim

What is claimed is:

1. A system to modify computer program output, comprising:
a data processing system having one or more processors and memory to:
receive, from a computing device, a digital file corresponding to a first acoustic signal carrying voice content detected by a microphone of the computing device;
select, responsive to the voice content of the digital file, a computer program comprising a chatbot from a plurality of computer programs comprising chatbots;
identify a dialog data structure generated by the chatbot responsive to the voice content of the digital file;
determine, based on the dialog data structure generated by the chatbot, to select content via a content selection process for provision with the dialog data structure;
select, via the content selection process, a content item for provision with the dialog data structure; and
provide, to the chatbot, the content item selected via the content selection process to cause the computing device to generate a second acoustic signal corresponding to the dialog data structure with the content item, wherein the computing device plays the content item with an acoustic fingerprint corresponding to the chatbot.

2. The system of claim 1 , comprising the data processing system to:
identify a tag in the dialog data structure generated by the chatbot; and
determine, responsive to the tag, to select content via the content selection process.

3. The system of claim 1 , comprising the data processing system to:
identify, based on the dialog data structure, an indication to generate a request for content; and
determine, responsive to the request, to select content via the content selection process.

4. The system of claim 1 , comprising the data processing system to:
generate, based on the dialog data structure, a request for content selected via the content selection process; and
determine, responsive to the request, to select content via the content selection process.

5. The system of claim 1 , comprising the data processing system to:
identify a placeholder field in the dialog data structure;
generate a request for content responsive to identification of the placeholder field; and
determine, responsive to the request, to select content via the content selection process.

6. The system of claim 5 , comprising the data processing system to:
insert the content item into the placeholder field of the dialog data structure.

7. The system of claim 1 , wherein the computing device plays the content item with the acoustic fingerprint matching the acoustic fingerprint the chatbot.

8. The system of claim 1 , wherein the content item comprises a parameterized format configured for a parametrically driven text to speech technique, and the computing device executes the parametrically driven text to speech technique to play the content item in the acoustic fingerprint corresponding to the chatbot.

9. The system of claim 1 , comprising the data processing system to:
select, responsive to a third acoustic signal, a second chatbot from the plurality of computer programs comprising chatbots;
select a second content item for insertion into a placeholder field in a second dialog data structure to be provided via the second chatbot; and
provide the second content item to the computing device to cause the computing device to play the second content item with a second acoustic fingerprint corresponding to the second chatbot, wherein the second acoustic fingerprint is different from the acoustic fingerprint corresponding to the chatbot.

10. The system of claim 1 , wherein each of the plurality of computer programs comprising chatbots is associated with a different acoustic fingerprint.

11. The system of claim 1 , comprising the data processing system to:
use a natural language processing technique to process the dialog data structure and identify a portion of the dialog data structure at which to insert content selected via the content selection process; and
provide an indication to insert content selected via the content selection process at the portion of the dialog data structure.

12. A method of modifying computer program output, comprising:
receiving, by a data processing system having one or more processors and memory, from a computing device, a digital file corresponding to a first acoustic signal carrying voice content detected by a microphone of the computing device;
selecting, by the data processing system responsive to the voice content of the digital file, a computer program comprising a chatbot from a plurality of computer programs comprising chatbots;
identifying, by the data processing system, a dialog data structure generated by the chatbot responsive to the voice content of the digital file;
determining, by the data processing system based on the dialog data structure generated by the chatbot, to select content via a content selection process for provision with the dialog data structure;
selecting, by the data processing system via the content selection process, a content item for provision with the dialog data structure; and
providing, by the data processing system to the chatbot, the content item selected via the content selection process to cause the computing device to generate a second acoustic signal corresponding to the dialog data structure with the content item, wherein the computing device plays the content item with an acoustic fingerprint corresponding to the chatbot.

13. The method of claim 12 , comprising:
identifying, by the data processing system, a tag in the dialog data structure generated by the chatbot; and
determining, by the data processing system responsive to the tag, to select content via the content selection process.

14. The method of claim 12 , comprising:
identifying, by the data processing system based on the dialog data structure, an indication to generate a request for content; and
determining, by the data processing system responsive to the request, to select content via the content selection process.

15. The method of claim 12 , comprising:
generating, by the data processing system based on the dialog data structure, a request for content selected via the content selection process; and
determining, by the data processing system responsive to the request, to select content via the content selection process.

16. The method of claim 12 , comprising:
identifying, by the data processing system, a placeholder field in the dialog data structure;
generating, by the data processing system, a request for content responsive to identification of the placeholder field; and
determining, by the data processing system responsive to the request, to select content via the content selection process.

17. The method of claim 16 , comprising:
inserting, by the data processing system, the content item into the placeholder field of the dialog data structure.

18. The method of claim 12 , wherein the computing device plays the content item with the acoustic fingerprint matching the acoustic fingerprint the chatbot.

19. The method of claim 12 , wherein the content item comprises a parameterized format configured for a parametrically driven text to speech technique, and the computing device executes the parametrically driven text to speech technique to play the content item in the acoustic fingerprint corresponding to the chatbot.

20. The method of claim 12 , comprising:
selecting, by the data processing system responsive to a third acoustic signal, a second chatbot from the plurality of computer programs

Assignees

• Google Llc

Inventors

• Eidem Laura
• Jacobson Alex

Classifications

• H04L67/306

  User profiles · CPC title

• G10L13/08

  Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination · CPC title

• G06F40/253

  Grammatical analysis; Style critique · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title