Attention-based machine learning techniques using temporal sequence data and dynamic co-occurrence graph data objects

Patent metadata

Field	Value
Publication number	US-2024062052-A1
Application number	US-202217820681-A
Country	US
Kind code	A1
Filing date	Aug 18, 2022
Priority date	Aug 18, 2022
Publication date	Feb 22, 2024
Grant date	—

Abstract

Various embodiments of the present invention provide methods, apparatus, systems, computing devices, computing entities, and/or the like for generating a representative embeddings for a plurality of temporal sequences by using a graph attention augmented temporal network based at least in part on dynamic co-occurrence graphs for preceding temporal sequences and initial embeddings, where the dynamic co-occurrence graphs are projections of a global guidance co-occurrence graph on features of the preceding temporal sequences, and the initial embeddings are generated by processing a latent representation of corresponding features that is generated by a sequential long short term memory model as well as a feature tree using a tree-based long short term memory model.

First claim

What is claimed is:

1 . A computer-implemented method for classification using a machine learning model, the computer-implemented method comprising:
receiving, by a computing device, one or more input data objects, each input data object comprising a temporal sequence in a plurality of temporal sequences and comprising a related feature subset of a plurality of features associated with the temporal sequence; generating, by the computing device, a global guidance correlation graph data object, wherein: (i) each node of the global guidance correlation graph data object corresponds to a feature in the plurality of features, and (ii) each edge of the global guidance correlation graph data object corresponds to a feature pair and describes a co-occurrence probability for the feature pair; for each temporal sequence, generating, by the computing device, one or more dynamic co-occurrence graph data object based at least in part on the global guidance correlation graph, wherein each dynamic co-occurrence graph data object for a particular temporal sequence describes a projection of the global guidance correlation graph data object on the input data object for the temporal sequence; generating, by the computing device, using the machine learning model, and based at least in part on the plurality of temporal sequences and each dynamic co-occurrence graph data object, one or more predicted classification labels, wherein:
the machine learning model comprises a graph-attention augmented temporal neural network machine learning model comprising a plurality of embedding layers,
training the machine learning model comprises, for each combination of a given temporal sequence t of T number of temporal sequences in the plurality of temporal sequences, a given non-initial embedding layer l of the one or more embedding layers, and a given feature i of the plurality of features, generating a historical node representation based at least in part on: (i) a prior-layer historical node representation for the given temporal sequence t and the given feature i as generated by a preceding embedding layer l−1, and (ii) neighbor nodes for a target node associated with the given feature i in the dynamic co-occurrence graph corresponding to the given temporal sequence t,
an initial embedding layer is configured to, for an initial temporal sequence, generate historical node representations for the plurality of features using a tree-of-sequences based at least in part on initial embeddings that are generated using a sequential long short-term memory machine learning model; and
performing one or more prediction-based actions based at least in part on the one or more predictive classification labels.

2 . The computer-implemented method of claim 1 , wherein each edge of the one or more dynamic co-occurrence graph data objects for a particular temporal sequence is associated with a respective feature pair that are both in the related feature subset for the particular temporal sequence.

3 . The computer-implemented method of claim 1 , wherein an initial embedding for a particular feature is generated based at least in part on a latent representation of text data associated with the particular feature and hidden representation of sequential long short-term memory machine learning models for one or more related features for the particular feature as defined by a classification tree of a tree-of-sequences long short-term memory machine learning model.

4 . The computer-implemented method of claim 1 , wherein the one or more predicted classification labels are generated based at least in part on a hidden state generated based at least in part on historical node representations for the related feature subset of a final temporal sequence.

5 . The computer-implemented method of claim 1 , wherein:
each dynamic co-occurrence graph comprises a sequence of adjacency matrices.

6 . The computer-implemented method of claim 1 , wherein the historical node representation for the given temporal sequence t, the given non-initial embedding layer l, and the given feature i is generated using operations of h {t,i} i =σ(Σ {j∈N i } α {ij} h {t,i} {l−1} W l +b l ), where σ comprises a non-linear activation function, W and b comprise learnable parameters, and N i comprises the neighbor nodes for the target node associated with the given feature i in the dynamic co-occurrence graph corresponding to the given temporal sequence t.

7 . The computer-implemented method of claim 1 , wherein the co-occurrence probability for a particular feature pair describes a count of co-occurrences of the particular feature pair in a common temporal sequence across all of the plurality of input data objects.

8 . An apparatus for classification using a machine learning model, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the processor, cause the apparatus to at least:
receive one or more input data objects, each input data object comprising a temporal sequence in a plurality of temporal sequences and comprising a related feature subset of a plurality of features associated with the temporal sequence; generate a global guidance correlation graph data object, wherein: (i) each node of the global guidance correlation graph data object corresponds to a feature in the plurality of features, and (ii) each edge of the global guidance correlation graph data object corresponds to a feature pair and describes a co-occurrence probability for the feature pair; for each temporal sequence, generate one or more dynamic co-occurrence graph data object based at least in part on the global guidance correlation graph, wherein each dynamic co-occurrence graph data object for a particular temporal sequence describes a projection of the global guidance correlation graph data object on the input data object for the temporal sequence; generate, using the machine learning model, and based at least in part on the plurality of temporal sequences and each dynamic co-occurrence graph data object, one or more predicted classification labels, wherein:
the machine learning model comprises a graph-attention augmented temporal neural network machine learning model comprising a plurality of embedding layers,
training the machine learning model comprises, for each combination of a given temporal sequence t of T number of temporal sequences in the plurality of temporal sequences, a given non-initial embedding layer l of the one or more embedding layers, and a given feature i of the plurality of features, generating a historical node representation based at least in part on: (i) a prior-layer historical node representation for the given temporal sequence t and the given feature i as generated by a preceding embedding layer l−1, and (ii) neighbor nodes for a target node associated with the given feature i in the dynamic co-occurrence graph corresponding to the given temporal sequence t,
an initial embedding layer is configured to, for an initial temporal sequence, generate historical node representations for the plurality of features using a tree-of-sequences based at least in part on initial embeddings that are generated using a sequential long short-term memory machine learning model; and
perform one or more prediction-based actions based at least in part on the one or more predictive classification labels.

9 . The apparatus of claim 8 , wherein each edge of the one or more dynamic co-occurrence graph data objects for a particular temporal sequence is associated with a respective feature pair that are both in the related feature subset for the particular temporal sequence.

10 . The apparatus

Assignees

• Optum Inc

Inventors

• Kumar Amit
• Roy Suman
• Sengupta Ayan
• Godden Paul J

Classifications

• G16H40/20

  for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms · CPC title

• G16H15/00

  ICT specially adapted for medical reports, e.g. generation or transmission thereof · CPC title

• G16H10/60

  for patient-specific data, e.g. for electronic patient records · CPC title

• G16H50/70

  for mining of medical data, e.g. analysing previous cases of other patients · CPC title

• G16H50/20

  for computer-aided diagnosis, e.g. based on medical expert systems · CPC title