Conversational assistant for conversational engagement

Patent metadata

Field	Value
Publication number	US-12367869-B2
Application number	US-202017613373-A
Country	US
Kind code	B2
Filing date	Jun 15, 2020
Priority date	Jul 18, 2019
Publication date	Jul 22, 2025
Grant date	Jul 22, 2025

Abstract

A conversational assistant for conversational engagement platform can contain various modules including a user-model augmentation module, a dialogue management module, and a user-state analysis input/output module. The dialogue management module receives metrics tied to a user from the other modules to understand a current topic and a user's emotions regarding the current topic from the user-state analysis input/output module and then adapts dialogue from the dialogue management module to the user based on dialogue rules factoring in these different metrics. The dialogue rules also factors in both i) a duration of a conversational engagement with the user and ii) an attempt to maintain a positive experience for the user with the conversational engagement. A flexible ontology relationship representation about the user is built and stores learned metrics about the user over time with each conversational engagement, and then in combination with the dialogue rules, drives the conversations with the user.

First claim

The invention claimed is:

1. A conversational assistant for conversational engagement platform, comprising:
a conversational engagement microservice platform containing various modules of a user-model augmentation module, a dialogue management module, and a user-state analysis input/output module, where the dialogue management module is configured to receive metrics tied to a user from the other modules to understand a current topic of a conversation and a user's emotions regarding the current topic of the conversation from the user-state analysis input/output module and then adapt dialogue from the dialogue management module to the user based on a first set of dialogue rules factoring in these different metrics, where the conversational engagement microservice platform containing the various modules is configured to at least use the user-state analysis input/output module to extract the metrics tied to the user through conversations with the user over multiple different periods of interaction, where the first set of dialogue rules of the conversational engagement microservice platform containing the various modules are also configured to factor both i) a duration of a conversational engagement with the user and ii) an attempt to maintain a positive experience for the user with the conversational engagement, where the dialogue management module cooperates with the user-model augmentation module to implement and maintain a flexible ontology relationship representation about the user that builds and stores learned metrics about the user over time with each conversational engagement and then in combination with the first set of dialogue rules, drive the conversations in the conversational engagement with the user, where the dialogue management module is configured to cooperate with an information-extraction and topic-understanding input/output module to receive i) the metrics tied to the user and ii) at least one of spoken words and a transcript of the conversational engagement with the user to deduce and understand the current topic of the conversation without requiring the user to provide advance notice of what the current topic of the conversation will be about,
one or more non-transitory computer readable mediums to store software implemented in the conversational engagement microservice platform, and
one or more processors configured to execute the stored software in the one or more non-transitory computer readable mediums.

2. The conversational assistant for conversational engagement platform of claim 1 , wherein the user-model augmentation module is further configured to detect for and track, via storing in long-term memory, user-specific knowledge on two or more topics selected from a group consisting of: 1) family and other people associated with this user, 2) concerns this user has expressed, 3) emotional state associated with this user, 4) auditorily-detectable health indicators associated with this user, and 5) interests associated with this user, as well as
wherein the user-model augmentation module is further configured to create and maintain one or more nodes for each of these topics in the flexible ontology relationship representation as well as link relationships between two or more topics tailored specific to the user.

3. The conversational assistant for conversational engagement platform of claim 2 , where each node in the flexible ontology relationship representation associated with this user is associated with a particular topic and has a link to a storage location of learned facts and other information about the topic associated with that node, where one or more of the topic nodes and the stored learned facts are referenced by the dialogue management module to populate questions and/or responses in the dialogue when the dialogue management module dialogues with the user.

4. The conversational assistant for conversational engagement platform of claim 1 , wherein the user-model augmentation module is further configured to use a second set of dialogue rules codified through a dialogue specification language to dynamically adapt between the conversational assistant for conversational engagement platform initiating i) a free-form casual conversation with the user and ii) a directed dialogue with the user on a specific topic, based on a current conversational context.

5. The conversational assistant for conversational engagement platform of claim 4 , wherein the dialogue specification language is configured to enable expressive context-aware modeling of conversational content for end users in a textual language, where the second set of dialogue rules and branching structures is further codified to decide whether to continue on the current topic and when to transition off that topic but continue an engagement conversation with another topic is codified in the dialogue specification language, where the dialogue specification language is structured to enable a code size reduction from one or more previous dialogue workflow specifications; and thus, saves space on memory storage and power consumption to use this dialogue specification language.

6. The conversational assistant for conversational engagement platform of claim 1 , wherein once a given topic is selected to be discussed, then the dialogue management module is configured to use the first set of dialogue rules to review i) an entire current conversation on that topic, ii) stored information on one or more previous conversations on that topic, and iii) stored information from linked topic nodes, as weighted factors to guide what a next question and/or response in the dialogue will be about; and then, utilize dialogue templates populated and derived from the above information to generate the dialogue.

7. The conversational assistant for conversational engagement platform of claim 1 , wherein the user-state analysis input/output module is configured to cooperate with the dialogue management module and its specification of the first set of dialogue rules in order to use a user state analysis to adapt the topic of the conversation to 1) points in that topic the user is amenable to and/or 2) points in another topic that the user is positive to and steer the conversation away from points in topics that the user state indicates negativity currently toward.

8. The conversational assistant for conversational engagement platform of claim 1 , further comprising:
where the information-extraction and topic-understanding input/output module is configured to cooperate with the dialogue management module, where the information-extraction and topic-understanding input/output module is configured to detect and keep track of a topic ID, derived from hierarchical classifiers for topics and co-clustering of related topics, in order to correctly identify a set of topics that are discussed in a free-form conversation between the user and the conversational assistant for conversational engagement platform.

9. The conversational assistant for conversational engagement platform of claim 1 , further comprising:
an audibly detectable health input/output module configured to cooperate with the dialogue management module, where the audibly detectable health input/output module is configured to i) monitor all dialogues with the user in order to monitor a current state of a user's health, and ii) reference data stored on a prior history of the user's health condition, including coughs, in a current or subsequent conversation session, when deciding when to change the current topic over to a topic on the user's health.

10. The conversational assistant for conversational engagement platform of claim 1 , further comprising:
a text-to-speech input/output module that has at least one of i) one or more i

Assignees

• Stanford Res Inst Int

Inventors

• Kalns Edgar T
• Vergyri Dimitra
• Acharya Girish
• Kathol Andreas
• Almada Leonor
• Kim Hyong-Gyun
• Basiou Nikoletta
• Wessel Michael
• Spaulding Aaron
• Heusser Roland
• Carpenter James F
• Yin Min

Classifications

• G10L25/66

  for extracting parameters related to health condition (detecting or measuring for diagnostic purposes A61B5/00) · CPC title

• G10L25/63

  for estimating an emotional state · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L13/02

  Methods for producing synthetic speech; Speech synthesisers · CPC title

• G06F40/35

  Discourse or dialogue representation · CPC title