Validate modification of audio-based computer program output

Patent metadata

Field	Value
Publication number	US-10657173-B2
Application number	US-201715618873-A
Country	US
Kind code	B2
Filing date	Jun 9, 2017
Priority date	Jun 9, 2017
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

Modifying computer program output in a voice or non-text input activated environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to identify a computer program to invoke. The computer program can identify a dialog data structure. The system can modify the identified dialog data structure to include a content item. The system can provide the modified dialog data structure to a computing device for presentation. The system can validate the dialog data structure output by the computing device for presentation.

First claim

What is claimed is:

1. A system to validate modification of computer program output, comprising:
a data processing system having one or more processors and memory to:
establish a communication channel with a third-party server that provides a computer program comprising a chatbot, the computer program comprising the chatbot selected based on an acoustic signal detected by a microphone of a computing device;
receive, from the third-party server, a request for content in a parameterized format configured for a parametrically driven text to speech technique, the request triggered by identification of a placeholder field in a dialog data structure identified by the chatbot;
select, via a content selection process responsive to the request, a content item for insertion into the placeholder field of the dialog data structure, the content item in the parameterized format configured for the parametrically driven text to speech technique;
transmit, to the third-party server different from the data processing system, the content item in the parameterized format selected via the content selection process for provision to the chatbot to cause the computing device to perform the parametrically driven text to speech technique to generate a second acoustic signal corresponding to the dialog data structure modified with the content item;
receive, from the chatbot, an indication of the content item;
determine, based on a comparison of the indication of the content item with the content item, that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server; and
set, based on the determination that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server, a validation parameter for the third-party server.

2. The system of claim 1 , comprising the data processing system to:
ping the chatbot subsequent to transmission of the content item; and
receive, from the chatbot, the indication of the content item responsive to the ping.

3. The system of claim 1 , comprising the data processing system to:
ping, subsequent to transmission of the content item, the chatbot based on a timing function; and
receive, from the chatbot, the indication of the content item responsive to the ping.

4. The system of claim 1 , comprising the data processing system to:
receive an indication of a second content item from the chatbot;
determine, based on a comparison, that the indication of the second content item received from the chatbot does not match the content item transmitted by the data processing system to the third-party server; and
set the validation parameter based on the determination.

5. The system of claim 1 , comprising the data processing system to:
receive an indication of a second content item from the chatbot;
determine, based on a comparison, that the indication of the second content item received from the chatbot does not match the content item transmitted by the data processing system to the third-party server; and
disable, based on the determination, provision of content items to chatbots provided by the third-party server.

6. The system of claim 1 , comprising the data processing system to:
determine, based on the comparison, that the indication of the content item received from the chatbot matches the content item transmitted by the data processing system to the third-party server; and
record, responsive to the determination, network activity information associated with an interaction of the content item via the computing device.

7. The system of claim 1 , comprising the data processing system to:
encrypt the content item with a hash function and salts to generate an encrypted content item; and
transmit the encrypted content item to the third-party server.

8. The system of claim 1 , comprising:
encrypt the content item with a hash function and salts to generate an encrypted content item; and
transmit the encrypted content item to the third-party server.

9. The system of claim 1 , comprising the data processing system to:
determine, based on an identifier of the computing device and via a lookup in a data repository, that the computing device is authorized to access the computer program comprising the chatbot; and
select the content item responsive to the determination that the computing device is authorized to access the chatbot.

10. A method of validating a modification of computer program output, comprising:
establishing, by a data processing system, a communication channel with a third-party server that provides a computer program comprising a chatbot, the computer program comprising the chatbot selected based on an acoustic signal detected by a microphone of a computing device;
receiving, by the data processing system from the third-party server, a request for content in a parameterized format configured for a parametrically driven text to speech technique, the request triggered by identification of a placeholder field in a dialog data structure identified by the chatbot;
selecting, by the data processing system via a content selection process responsive to the request, a content item for insertion into the placeholder field of the dialog data structure, the content item in the parameterized format configured for the parametrically driven text to speech technique;
transmitting, by the data processing system to the third-party server, the content item in the parameterized format selected via the content selection process for provision to the chatbot to cause the computing device to perform the parametrically driven text to speech technique to generate a second acoustic signal corresponding to the dialog data structure modified with the content item;
receiving, by the data processing system from the chatbot, an indication of the content item;
determining, by the data processing system based on a comparison of the indication of the content item with the content item, that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server; and
setting, by the data processing system, based on the determination that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server, a validation parameter for the third-party server.

11. The method of claim 10 , comprising:
pinging the chatbot subsequent to transmission of the content item; and
receiving, from the chatbot, the indication of the content item responsive to the pinging.

12. The method of claim 10 , comprising:
pinging, subsequent to transmission of the content item, the chatbot based on a timing function; and
receiving, from the chatbot, the indication of the content item responsive to the pinging.

13. The method of claim 10 , comprising:
receiving, by the data processing system, an indication of a second content item from the chatbot;
determining, based on a comparison, that the indication of the second content item received from the chatbot does not match the content item transmitted by the data processing system to the third-party server; and
setting the validation parameter based on the determination.

14. The method of claim 10 , comprising:
receiving, by the data processing system, an indication of a second content item from the chatbot;
determining, based on a comparison, that the indication of the second content item received from the chatb

Assignees

• Google Inc
• Google Llc

Inventors

• Eidem Laura
• Jacobson Alex

Classifications

• H04L63/0457

  wherein the sending and receiving network entities apply dynamic encryption, e.g. stream encryption (cryptographic mechanisms or cryptographic arrangements for stream encryption H04L9/065) · CPC title

• H04L63/0823

  using certificates (cryptographic mechanisms or cryptographic arrangements for entity authentication involving certificates H04L9/3263) · CPC title

• G06F16/61

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/683 Primary

  using metadata automatically derived from the content · CPC title

• H04L51/02

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title