Method and apparatus for building an intelligent automated assistant

Patent metadata

Field	Value
Publication number	US-11928604-B2
Application number	US-201916379048-A
Country	US
Kind code	B2
Filing date	Apr 9, 2019
Priority date	Sep 8, 2005
Publication date	Mar 12, 2024
Grant date	Mar 12, 2024

Abstract

A method and apparatus are provided for building an intelligent automated assistant. Embodiments of the present invention rely on the concept of “active ontologies” (e.g., execution environments constructed in an ontology-like manner) to build and run applications for use by intelligent automated assistants. In one specific embodiment, a method for building an automated assistant includes interfacing a service-oriented architecture that includes a plurality of remote services to an active ontology, where the active ontology includes at least one active processing element that models a domain. At least one of the remote services is then registered for use in the domain.

First claim

What is claimed is:

1. A computing device, comprising:
one or more processors;
memory; and
one or more programs stored in memory, the one or more programs including instructions for:
collecting at least one input corresponding to one or more events associated with an operating environment of the computing device;
obtaining priorities associated with a plurality of active processing elements included in an active ontology, of an intelligent automated assistant, each active processing element being configured to perform one or more actions responsive to receiving one or more inputs of the collected at least one input;
processing, based on the obtained priorities, the collected at least one input using the plurality of active processing elements;
identifying at least one action based on results of the plurality of active processing elements processing the collected at least one input, wherein identifying the at least one action comprises, identifying based on the processing results, one or more actions associated with valid conditions; and
executing the identified at least one action.

2. The computing device of claim 1 , wherein collecting at least one input comprises:
receiving the at least one input via one or more sensors distributed in the operating environment of the computing device.

3. The computing device of claim 1 , wherein collecting at least one input comprises:
receiving the at least one input via a multimodal user interface.

4. The computing device of claim 1 , wherein the at least one action comprises at least one of: activity recognition, time recognition, natural language processing, reactive activity execution, rule-based planning, automated scheduling, task automation, temporal constraint management, and service brokering.

5. The computing device of claim 1 , wherein the at least one input is tagged with an associated context and a current execution pass.

6. The computing device of claim 1 , wherein processing, based on the obtained priorities, the collected at least one input using the plurality of active processing elements comprises:
processing the collected at least one input in accordance with an order of priority associated with the plurality of active processing elements, wherein the order of priority is from highest priority to the lowest priority.

7. The computing device of claim 6 , wherein processing the collected at least one input in accordance with an order of priority associated with the plurality of active processing elements comprises:
for each active processing element of the plurality of active processing elements, processing the collected at least one input using a plurality of rules of the corresponding active processing element in accordance with an order of priority associated with the plurality of rules.

8. The computing device of claim 7 , wherein each rule of the plurality of rules comprises a condition and an associated action.

9. The computing device of claim 8 , wherein processing the collected at least one input using a plurality of rules comprises:
determining whether the collected at least one input match with a condition associated with each rule of the plurality of rules.

10. The computing device of claim 1 , wherein processing, based on the obtained priorities, the collected at least one input using the plurality of active processing elements comprises:
sharing information among at least two of the plurality of active processing elements.

11. The computing device of claim 10 , wherein sharing information among at least two of the plurality of active processing elements comprises:
propagating information from a first active processing element to a second active processing element, wherein the information propagated is weighted or associated with a probability.

12. The computing device of claim 1 , wherein the plurality of active processing elements is associated with a specific context.

13. The computing device of claim 1 , wherein the plurality of active processing elements included in the active ontology are arranged in an ontology-like manner such that each active processing element of the plurality of active processing elements is communicatively coupled to at least one other active processing element.

14. The computing device of claim 13 , wherein active processing elements that are communicatively coupled share processing results with one another.

15. The computing device of claim 1 , wherein processing the collected at least one input using the plurality of active of active processing elements is further based on a predefined order of priority.

16. A method for processing inputs in accordance with an active ontology, comprising:
at a computing device including one or more processors and a memory:
collecting at least one input corresponding to one or more events associated with an operating environment of the computing device;
obtaining priorities associated with a plurality of active processing elements included in the active ontology of an intelligent automated assistant, each active processing element being configured to perform one or more actions responsive to receiving one or more inputs of the collected at least one input;
processing, based on the obtained priorities, the collected at least one input using the plurality of active processing elements;
identifying at least one action based on results of the plurality of active processing elements processing the collected at least one input, wherein identifying the at least one action comprises, identifying based on the processing results, one or more actions associated with valid conditions; and
executing the identified at least one action.

17. The method of claim 16 , wherein collecting at least one input comprises:
receiving the at least one input via one or more sensors distributed in the operating environment of the computing device.

18. The method of claim 16 , wherein collecting at least one input comprises:
receiving the at least one input via a multimodal user interface.

19. The method of claim 16 , wherein the at least one action comprises at least one of:
activity recognition, time recognition, natural language processing, reactive activity execution, rule-based planning, automated scheduling, task automation, temporal constraint management, and service brokering.

20. The method of claim 16 , wherein the at least one input is tagged with an associated context and a current execution pass.

21. The method of claim 16 , wherein processing, based on the obtained priorities, the collected at least one input using the plurality of active processing elements comprises:
processing the collected at least one input in accordance with an order of priority associated with the plurality of active processing elements, wherein the order of priority is from highest priority to the lowest priority.

22. The method of claim 21 , wherein processing the collected at least one input in accordance with an order of priority associated with the plurality of active processing elements comprises:
for each active processing element of the plurality of active processing elements, processing the collected at least one input using a plurality of rules of the corresponding active processing element in accordance with an order of priority associated with the plurality of rules.

23. The method of claim 22 , wherein each rule of the plurality of rules comprises a condition and an associated action.

24. The method

Assignees

• Apple Inc

Inventors

• Cheyer Adam
• Guzzoni Didier

Classifications

• G06N5/022 Primary

  Knowledge engineering; Knowledge acquisition · CPC title

• G06F9/54

  Interprogram communication · CPC title

• G06F9/542

  Event management; Broadcasting; Multicasting; Notifications · CPC title

• G06N5/02

  Knowledge representation; Symbolic representation · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title