Activation of remote devices in a networked system

Patent metadata

Field	Value
Publication number	US-11514907-B2
Application number	US-202016860679-A
Country	US
Kind code	B2
Filing date	Apr 28, 2020
Priority date	May 7, 2018
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

The present disclosure is generally directed to the generation of voice-activated data flows in interconnected network. The voice-activated data flows can include input audio signals that include a request and are detected at a client device. The client device can transmit the input audio signal to a data processing system, where the input audio signal can be parsed and passed to the data processing system of a service provider to fulfill the request in the input audio signal. The present solution is configured to conserve network resources by reducing the number of network transmissions needed to fulfill a request.

First claim

The invention claimed is:

1. A system to generate voice-activated threads in a networked computer environment, comprising:
a natural language processor component executed by a data processing system to:
receive first input audio signal detected by a sensor of a first client computing device;
parse the first input audio signal to identify a first request;
parse the first input audio signal to identify a response entity in the first input audio signal;
a digital component selector executed by the data processing system to select a digital component provided by a digital component provider device based on the first request, and transmit the digital component to the first client computing device, wherein the digital component comprises an indication of an application associated with a service provider device and the response entity;
an action handler component executed by the data processing system to select a first action data structure based on the first request, the first action data structure associated with a first service provider device;
the action handler component to:
convert the response entity based on an expansion policy associated with the first client computing device and into a format associated with a field in the first action data structure;
store the response entity in the format associated with the field in the first action data structure; and
an interface of the data processing system to transmit the first action data structure to the first service provider device to fulfill the first request.

2. The system of claim 1 , comprising:
the digital component selector to transmit the digital component to a local digital assistant executing on the first client computing device.

3. The system of claim 1 , comprising:
the digital component selector to transmit the digital component to a third-party application executed by the first client computing device.

4. The system of claim 1 , comprising:
the digital component selector to select the digital component via a real-time content selection process that comprises determines a score and a rank for each of a plurality of digital components to cause selection of a highest scoring and ranking digital component of the plurality of digital components.

5. The system of claim 1 , comprising:
the digital component selector to select the digital component configured to fulfill the first request, wherein the digital component is different from the first action data structure.

6. The system of claim 1 , comprising:
the digital component selector to select the digital component associated with a second service provider device different from the first service provider device, wherein the digital component is configured to fulfill the first request.

7. The system of claim 1 , comprising:
the action handler component to transmit a first audio-based input request to the first client computing device based at least on a second field in the first action data structure;
the natural language processor component to receive a second input audio signal detected by the sensor of the first client computing device and generated in response to the first audio-based input request;
the natural language processor component to parse the second input audio signal to identify a second response entity in the second input audio signal; and
the action handler component to populate the second field in the first action data structure with the second response entity.

8. The system of claim 1 , comprising:
the natural language processor component to receive a second input audio signal detected by the sensor of the first client computing device;
the natural language processor component to parse the second input audio signal to identify a second request;
the action handler component to select a second action data structure based on the second request, the second action data structure associated with the first service provider device;
the natural language processor component to parse, the second input audio signal to identify a second response entity in the second input audio signal; and
the action handler component to determine not to expand the second response entity based on the expansion policy associated with the first client computing device.

9. The system of claim 1 , comprising:
the natural language processor component to map the response entity to the field in the first action data structure.

10. The system of claim 1 , comprising:
the action handler component to determine that the response entity comprises a second format not associated with the format associated with the field in the first action data structure; and
the action handler component to expand the response entity based on the response entity having the second format not associated with the format associated with the field in the first action data structure.

11. A method of generating voice-activated threads in a networked computer environment, comprising:
receiving, by a data processing system, first input audio signal detected by a sensor of a first client computing device;
parsing, by the data processing system, the first input audio signal to identify a first request;
parsing, by the data processing system, the first input audio signal to identify a response entity in the first input audio signal;
selecting, by the data processing system, a digital component provided by a digital component provider device based on the first request, wherein the digital component comprises an indication of an application associated with a service provider device and the response entity;
transmitting, by the data processing system, the digital component to the first client computing device;
selecting, by the data processing system, a first action data structure based on the first request, the first action data structure associated with a first service provider device;
converting, by the data processing system, the response entity based on an expansion policy associated with the first client computing device and into a format associated with a field in the first action data structure;
storing, by the data processing system, the response entity in the format associated with the field in the first action data structure; and
transmitting, by the data processing system, the first action data structure to the first service provider device to fulfill the first request.

12. The method of claim 11 , comprising:
transmitting, by the data processing system, the digital component to a local digital assistant executing on the first client computing device.

13. The method of claim 11 , comprising:
transmitting, by the data processing system, the digital component to a third-party application executed by the first client computing device.

14. The method of claim 11 , comprising:
selecting, by the data processing system, the digital component via a real-time content selection process that comprises determines a score and a rank for each of a plurality of digital components to cause selection of a highest scoring and ranking digital component of the plurality of digital components.

15. The method of claim 11 , comprising:
selecting, by the data processing system, the digital component configured to fulfill the first request, wherein the digital component is different from the first action data structure.

16. The method of claim 11 , comprising:
selecting, by the data processing system, the digital component associated with a second service provider device different from the first service provider device, wherein the digital component is configured to fulfill the first request.

Assignees

• Google Llc

Inventors

• Bhaya Gaurav
• Kirazci Ulas
• Abrams Bradley
• Coimbra Adam
• Firman Ilya
• Radebaugh Carey

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G06F40/35

  Discourse or dialogue representation · CPC title

• G10L2015/225

  Feedback of the input speech · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title