Interpretation of deep neural model based dialogue agents

Patent metadata

Field	Value
Publication number	US-11748576-B2
Application number	US-202117154640-A
Country	US
Kind code	B2
Filing date	Jan 21, 2021
Priority date	Jan 21, 2021
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

Techniques facilitating interpretation of deep neural model based dialogue agents are provided. In one example, a computer-implemented method comprises extracting, by a device operatively coupled to a processor, features from a dialogue model independently from the dialogue model, the features comprising input features provided to the dialogue model and output features produced via the dialogue model in response to the input features, resulting in extracted features; and analyzing, by the device, a dialogue context associated with the extracted features by identifying pairwise interactions between respective ones of the extracted features.

First claim

What is claimed is:

1. A system comprising:
a memory that stores computer executable components; and
a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise:
a feature extraction component that extracts features from a dialogue model without direct access to operating parameters of the dialogue model and wherein results of the dialogue model are interpreted in a model-agnostic manner enabling processing of the dialogue model for which inner workings of the dialogue model are shielded from access by the feature extraction component; and
a model interpreter component that analyzes a dialogue context associated with the features extracted by the feature extraction component by identifying pairwise interactions between respective ones of the features, wherein the features comprise input features and output features, wherein the input features are provided to the dialogue model and wherein the output features are produced via the dialogue model.

2. The system of claim 1 , wherein the computer executable components further comprise:
a feature constructor component that generates visual representations of the pairwise interactions between selected ones of the features and facilitates respectively rendering the visual representations on a display device, resulting in improved accuracy of automation tasks performed on a computing system via the dialogue model.

3. The system of claim 2 , wherein the computer executable components further comprise:
a feature aggregator component that computes interaction scores for respective ones of the pairwise interactions identified by the model interpreter component based on a feature significance criterion; and
a feature selector component that selects the selected ones of the features based on the interaction scores computed by the feature aggregator component.

4. The system of claim 3 , wherein the feature significance criterion comprises a criterion selected from a group consisting of role significance, dialogue turn significance, dialogue token significance, and token pair relationship significance.

5. The system of claim 3 , wherein the pairwise interactions comprise an interaction between a first feature and a second feature of the features as selected from a group comprising a negation of the first feature by the second feature and anaphora exhibited by the first feature and the second feature.

6. The system of claim 5 , wherein the visual representations generated by the feature constructor component comprise visual emphasis of the first feature and the second feature and an annotation of the interaction between the first feature and the second feature.

7. The system of claim 3 , wherein the computer executable components further comprise:
a user feedback component that receives user preferences corresponding to the visual representations, wherein the feature selector component selects the selected ones of the features further based on the user preferences.

8. The system of claim 2 , wherein the feature constructor component facilitates respectively rendering the visual representations as an overlay to displayed output features rendered via the dialogue model.

9. A computer-implemented method comprising:
extracting, by a device operatively coupled to a processor, features from a dialogue model without direct access to operating parameters of the dialogue model, resulting in extracted features; and
analyzing, by the device, a dialogue context associated with the extracted features by identifying pairwise interactions between respective ones of the extracted features, wherein the features comprise input features and output features, wherein the input features are provided to the dialogue model and wherein the output features are produced via the dialogue model.

10. The computer-implemented method of claim 9 , further comprising:
generating, by the device, visual representations of the pairwise interactions between selected features of the extracted features; and
facilitating, by the device, respectively rendering the visual representations on a display device, resulting in improved accuracy of automation tasks performed on a computing system via the dialogue model.

11. The computer-implemented method of claim 10 , further comprising:
computing, by the device, interaction scores for respective ones of the pairwise interactions between the respective ones of the extracted features based on a feature significance criterion; and
selecting, by the device, the selected features of the extracted features based on the interaction scores.

12. The computer-implemented method of claim 11 , wherein the feature significance criterion comprises a criterion selected from a group consisting of role significance, dialogue turn significance, dialogue token significance, and token pair relationship significance.

13. The computer-implemented method of claim 11 , wherein the pairwise interactions between the respective ones of the extracted features comprise an interaction between a first feature and a second feature of the extracted features as selected from a group comprising a negation of the first feature by the second feature and anaphora exhibited by the first feature and the second feature.

14. The computer-implemented method of claim 11 , further comprising:
receiving, by the device, user preferences corresponding to the visual representations; and
selecting, by the device, the selected features further based on the user preferences.

15. A computer program product for interpreting a deep neural model-based dialogue agent, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
extract features from a dialogue model without direct access to operating parameters of the dialogue model, resulting in extracted features; and
analyze a dialogue context associated with the extracted features by identifying pairwise interactions between respective ones of the extracted features, wherein the features comprise input features and output features, wherein the input features are provided to the dialogue model and wherein the output features are produced via the dialogue model.

16. The computer program product of claim 15 , wherein the program instructions further cause the processor to:
generate visual representations of the pairwise interactions between selected features of the extracted features; and
facilitate respectively rendering the visual representations on a display device, resulting in improved accuracy of automation tasks performed on a computing system via the dialogue model.

17. The computer program product of claim 16 , wherein the program instructions further cause the processor to:
compute interaction scores for respective ones of the pairwise interactions between the respective ones of the extracted features based on a feature significance criterion; and
select the selected features of the extracted features based on the interaction scores.

18. The computer program product of claim 17 , wherein the feature significance criterion comprises a criterion selected from a group consisting of role significance, dialogue turn significance, dialogue token significance, and token pair relationship significance.

19. The computer program product of claim 17 , wherein the pairwise interactions between the respective ones of the extracted features comprise an interactio

Assignees

• IBM

Inventors

• Feng Song
• Wan Hui
• Gunasekara Chulaka
• Ganhotra Jatin
• Joshi Sachindra
• Lastras-Montano Luis A

Classifications

• G06F40/35 Primary

  Discourse or dialogue representation · CPC title

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F40/44

  Statistical methods, e.g. probability models · CPC title