Action recipes for a crowdsourced digital assistant system

Patent metadata

Field	Value
Publication number	US-12093707-B2
Application number	US-202217750957-A
Country	US
Kind code	B2
Filing date	May 23, 2022
Priority date	May 18, 2017
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

Embodiments of the present invention are directed to action recipes for a crowdsourced digital assistant. Users can define an action recipe by recording a set of inputs across one or more applications, by providing multiple sub-commands in a single on-the-fly command, by providing one or more associated commands, or otherwise. An action recipe dataset is generated, and stored and indexed on a user device and/or on an action cloud server. As such, any user can invoke an action recipe by providing an associated command to a crowdsourced digital assistant application on a user device. The crowdsourced digital assistant searches for a matching command on the user device and/or the action cloud server, and if a match is located, the corresponding action recipe dataset is accessed, and the crowdsourced digital assistant emulates the actions in the action recipe on the user device.

First claim

What is claimed is:

1. A computer-implemented method for defining action recipes performed by a digital assistant application, the method comprising:
receiving, by a computing device, an instruction to generate an action recipe dataset that defines a sequence of actions associated with multiple applications;
detecting, by the computing device, a set of inputs into the multiple applications, wherein the set of inputs corresponds to the sequence of actions;
recording the detected set of inputs into the multiple applications; and
generating, by the computing device, the action recipe dataset based on the recorded set of inputs,
wherein the digital assistant application is configured to emulate the set of inputs in response to a received command,
wherein the digital assistant application is configured to pass data from a first application of the multiple applications to a second, different, application of the multiple applications when emulating the set of inputs in response to the received command,
wherein the digital assistant application is configured to provide the action recipe dataset to a remote server to facilitate distribution to multiple computing devices that each have the digital assistant application installed thereon, and
wherein the received command instructs the digital assistant application to:
utilize a representation of an output of the first application as a value of a variable input parameter in an input command to the second application when emulating the set of inputs in response to the received command,
apply a transformation to the output of the first application to generate transformed data, and
pass the transformed data as the value of the variable input parameter in the input command to the second application.

2. The computer-implemented method of claim 1 , wherein the digital assistant application is configured to pass the representation of the output from the first application as the value of the variable input parameter in the input command to the second application.

3. The computer-implemented method of claim 1 , wherein the action recipe dataset is configured to bind a first action of the sequence of actions to a particular function.

4. The computer-implemented method of claim 3 , wherein the digital assistant application is configured to emulate the first action by performing the function using an application selected from alternative applications.

5. The computer-implemented method of claim 1 , wherein the action recipe dataset is configured to dynamically replace an application.

6. The computer-implemented method of claim 1 , wherein the action recipe dataset is configured to pass data between actions of the sequence of actions.

7. The computer-implemented method of claim 1 , wherein the received command includes a second value of a second variable input parameter of an input into one of the multiple applications.

8. The non-transitory computer-readable medium of claim 1 , wherein the action recipe dataset is configured to dynamically replace an application.

9. The non-transitory computer-readable medium of claim 1 , wherein the action recipe dataset is configured to pass data between actions of the sequence of actions.

10. The non-transitory computer-readable medium of claim 1 , wherein the received command includes a second value of a second input parameter of an input into one of the multiple applications.

11. A non-transitory computer-readable medium whose contents, when executed by a computing device, cause the computing device to perform a method for defining action recipes performed by a digital assistant application, the method comprising:
receiving an instruction to generate an action recipe dataset that defines a sequence of actions associated with multiple applications;
detecting a set of inputs into the multiple applications, wherein the set of inputs corresponds to the sequence of actions;
recording the detected set of inputs into the multiple applications; and
generating the action recipe dataset based on the recorded set of inputs,
wherein the digital assistant application is configured to emulate the set of inputs in response to a received command,
wherein the digital assistant application is configured to pass data from a first application of the multiple applications to a second, different, application of the multiple applications when emulating the set of inputs in response to the received command,
wherein the digital assistant application is configured to provide the action recipe dataset to a remote server to facilitate distribution to multiple computing devices that each have the digital assistant application installed thereon, and
wherein the received command instructs the digital assistant application to:
utilize a representation of an output of the first application as a value of a variable input parameter in an input command to the second application when emulating the set of inputs in response to the received command,
apply a transformation to the output of the first application to generate transformed data, and
pass the transformed data as the value of the variable input parameter in the input command to the second application.

12. The non-transitory computer-readable medium of claim 11 , wherein the digital assistant application is configured to pass the representation of the output from the first application as the value of the input parameter in the input command to the second application.

13. The non-transitory computer-readable medium of claim 11 , wherein the action recipe dataset is configured to bind a first action of the sequence of actions to a particular function.

14. The non-transitory computer-readable medium of claim 13 , wherein the digital assistant application is configured to emulate the first action by performing the function using an application selected from alternative applications.

Assignees

• Peloton Interactive Inc

Inventors

• Mukherjee Rajat
• Robinson Mark
• Bindhu Hemaraj Kiran

Classifications

• G06F9/541

  via adapters, e.g. between incompatible applications · CPC title

• G06F9/5038

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/048

  Interaction techniques based on graphical user interfaces [GUI] · CPC title

• G06F9/45512 Primary

  Command shells · CPC title