Secure digital assistant integration in web pages

Patent metadata

Field	Value
Publication number	US-11829435-B2
Application number	US-202217837914-A
Country	US
Kind code	B2
Filing date	Jun 10, 2022
Priority date	May 6, 2019
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

Secure digital assistant integration with web pages is provided. The system receives an intent manifest data structure that maps actions of a digital assistant with link templates of an electronic resource developed by a third-party developer device. The system validates the electronic resource based on the intent manifest data structure. The system receives, from a data exchange component of an iframe of the electronic resource loaded by a client computing device, an identifier of the client computing device. The system receives a foreground state of the electronic resource from an onsite state sharing API. The system selects a data value for a parameter based on the foreground state and the intent manifest data structure. The system provides the data value. An authorization component generates an authorization prompt, receives input, and transmits the data value to an onsite intent execution API of the electronic resource to execute an action.

First claim

What is claimed is:

1. A system for secure digital assistant integration with web pages, comprising:
a client computing device comprising at least one processor to:
provide, to a data processing system, by a data exchange component of an electronic resource loaded by the client computing device, an identifier of the client computing device that executes the electronic resource;
provide, to the data processing system, a foreground state of the electronic resource from an onsite state sharing application programming interface;
receive, from the data processing system, by an authorization component of the electronic resource loaded by the client computing device, a data value for a parameter, the parameter being determined based on the foreground state and an intent manifest data structure containing a mapping between a plurality of actions of a digital assistant and a plurality of link templates of the electronic resource, and the data value being selected based on the identifier of the client computing device, the data value to cause the authorization component to:
generate an authorization prompt;
receive, responsive to the authorization prompt, input from the client computing device; and
transmit, responsive to authorization of the data value, the data value to an onsite intent execution application programming interface of the electronic resource to cause the electronic resource to execute an action of the plurality of actions with the data value.

2. The system of claim 1 , wherein the data exchange component of the electronic resource restricts the electronic resource from accessing the identifier of the client computing device, and a third-party developer device that developed the electronic resource is prohibited from accessing the identifier of the client computing device.

3. The system of claim 1 , wherein the data exchange component is authorized, by the data processing system, to load in the electronic resource responsive to validation of the electronic resource via a validation policy.

4. The system of claim 1 , comprising the client computing device to send, to the data processing system, by the data exchange component, a request, the data processing system querying the onsite state sharing application programming interface of the electronic resource responsive to the request.

5. The system of claim 1 , comprising the client computing device to:
send, to the data processing system, from a voice navigator and response component executed by the client computing device, data packets carrying an input audio signal detected by a sensor of the client computing device, the data packets causing the data processing system to identify a request for a candidate data value; and
receive, from the data processing system, the data value as the candidate data value responsive to the request.

6. The system of claim 1 , comprising the client computing device to:
receive, by the onsite intent execution application programming interface, the data value; and
input, by the onsite intent execution application programming interface, the data value into an input text box of the electronic resource.

7. The system of claim 1 , comprising the client computing device to:
receive, by the authorization component, a plurality of data values corresponding to a plurality of parameters used to execute the action provided by the electronic resource, the plurality of data values being selected based on the identifier of the client computing device, the plurality of parameters being determined based on the foreground state; and
provide, by the authorization component, the plurality of data values to the onsite intent execution application programming interface, wherein the onsite intent execution application programming interface is configured to use the plurality of data values to bypass one or more states used by the electronic resource to execute the action.

8. The system of claim 1 , comprising the client computing device to receive, prior to the electronic resource requesting the data value, the data value for authorization by the authorization component and input to the onsite intent execution application programming interface.

9. The system of claim 1 , comprising the client computing device to build a deep link with the data value, and load the deep link in a web browser executed by the client computing device.

10. The system of claim 1 , wherein the electronic resource comprises a web page.

11. The system of claim 1 , comprising the client computing device to provide, via the data exchange component, a link with the data value to the onsite intent execution application programming interface, the link being built based on a link template of the plurality of link templates that maps to the action of the plurality of actions.

12. A method for secure digital assistant integration with web pages, comprising:
providing, to a data processing system, by a data exchange component of an electronic resource loaded by a client computing device, an identifier of the client computing device that executes the electronic resource;
providing, to the data processing system, a foreground state of the electronic resource from an onsite state sharing application programming interface; and
receiving, from the data processing system, by an authorization component of the electronic resource loaded by the client computing device, a data value for a parameter, the parameter being determined based on the foreground state and an intent manifest data structure containing a mapping between a plurality of actions of a digital assistant and a plurality of link templates of the electronic resource, and the data value being selected based on the identifier of the client computing device, the data value to cause the authorization component to:
generate an authorization prompt;
receive, responsive to the authorization prompt, input from the client computing device; and
transmit, responsive to authorization of the data value, the data value to an onsite intent execution application programming interface of the electronic resource to cause the electronic resource to execute an action of the plurality of actions with the data value.

13. The method of claim 12 , wherein the data exchange component of the electronic resource restricts the electronic resource from accessing the identifier of the client computing device, and a third-party developer device that developed the electronic resource is prohibited from accessing the identifier of the client computing device.

14. The method of claim 12 , wherein the data exchange component is authorized, by the data processing system, to load in the electronic resource responsive to validation of the electronic resource via a validation policy.

15. The method of claim 12 , comprising sending, to the data processing system, by the data exchange component, a request, the data processing system querying the onsite state sharing application programming interface of the electronic resource responsive to the request.

16. The method of claim 12 , comprising:
sending, to the data processing system, from a voice navigator and response component executed by the client computing device, data packets carrying an input audio signal detected by a sensor of the client computing device, the data packets causing the data processing system to identify a request for a candidate data value; and
receiving, from the data processing system, the data value as the candidate data value responsive to the request.

17. The method of claim 12 , comprising:
receiving, by the onsite intent execution appli

Assignees

• Google Llc

Inventors

• Coimbra Adam
• Rao Nikhil
• Wu Hao
• Baker Ian
• Gupta Anshul
• Akkad Jay

Classifications

• G06F16/957 Primary

  Browsing optimisation, e.g. caching or content distillation · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F9/44589

  Program code verification, e.g. Java bytecode verification, proof-carrying code (high-level semantic checks G06F8/43; prevention of errors by analysis, debugging or testing of software G06F11/36) · CPC title

• G06F9/453

  Help systems · CPC title

• G06F9/468

  Specific access rights for resources, e.g. using capability register · CPC title