Time-factored performance prediction

Patent metadata

Field	Value
Publication number	US-11361244-B2
Application number	US-201816004096-A
Country	US
Kind code	B2
Filing date	Jun 8, 2018
Priority date	Jun 8, 2018
Publication date	Jun 14, 2022
Grant date	Jun 14, 2022

Abstract

Training query intents are allocated for multiple training entities into training time intervals in a time series based on a corresponding query intent time for each training query intent. Training performance results for the multiple training entities are allocated into the training time intervals in the time series based on a corresponding performance time of each training performance result. A machine learning model for a training milestone of the time series is trained based on the training query intents allocated to a training time interval prior to the training milestone and the training performance results allocated to a training time interval after the training milestone. Target performance for the target entity for an interval after a target milestone in the time series is predicted by inputting to the trained machine learning model target query intents allocated to the target entity in a target time interval before the target milestone.

First claim

What is claimed is:

1. A method of predicting performance of a target entity using a machine learning model, the method comprising:
allocating training query intents generated from a query-URL click graph for multiple training entities into training time intervals in a time series based on a corresponding query intent time for each training query intent, the training time intervals in the time series being separated by training milestones in the time series, wherein the training milestones represent occurrences of predefined types of temporally related events that influence performance of the training entities;
allocating training performance results for the multiple training entities into the training time intervals in the time series based on a corresponding performance time of each training performance result;
selecting one or more query intents from the training query intents allocated to a training time interval prior to a time of one of the training milestones;
selecting one or more training performance results from the allocated training performance results after the time of the one of the training milestones;
training the machine learning model for the one of the training milestones of the time series based on the selected one or more training query intents allocated to a training time interval prior to the time of the one of the training milestones and the selected one or more training performance results allocated to a training time interval after the time of the one of the training milestones;
allocating target query intents generated from the query-URL click graph for the target entity into target time intervals based on a corresponding query intent time for each target query intent, the target time intervals in the time series being separated by target milestones in the time series, wherein the target milestones represent occurrences of predefined types of temporally related events that influence performance of the target entities; and
generating a prediction of a target performance result for the target entity for an interval after a target milestone in the time series by inputting to the trained machine learning model target query intents allocated to the target entity in a target time interval before the target milestone.

2. The method of claim 1 wherein the training query intents are generated from the query-URL click graph by a generating operation comprising:
selecting the multiple training entities that satisfy a training entity selection condition relative to the target entity;
extracting from the query-URL click graph training queries associated with the multiple training entities;
embedding the training queries extracted from the query-URL click graph into multidimensional vectors; and
clustering the multidimensional vectors into multiple groups, the multidimensional vectors of a first group having greater similarity amongst themselves as defined by a similarity condition for the first group than the multidimensional vectors of a second group, each group of multidimensional records constituting one of the training query intents.

3. The method of claim 1 wherein the target query intents are generated from the query-URL click graph by a generating operation comprising:
extracting from the query-URL click graph target queries associated with the target entity;
embedding the target queries extracted from the query-URL click graph into multidimensional vectors; and
clustering the multidimensional vectors into multiple groups, the multidimensional vectors of a first group having greater similarity amongst themselves as defined by a similarity condition for the first group than the multidimensional vectors of a second group, each group of multidimensional records constituting one of the target query intents.

4. The method of claim 1 wherein the training query intents are generated from a query-URL click graph by a generating operation comprising:
distributing the training query intents in the time series based on time values associated with the training queries.

5. The method of claim 1 wherein the operations of allocating training query intents and allocating training performance results are based on metadata extracted from an entity knowledge graph.

6. The method of claim 1 wherein the training operation comprises:
training the machine learning model for a different training milestone of the time series based on the allocated training query intents allocated to a different training interval prior to the training milestone and the allocated training performance results allocated to a different training interval after the training milestone.

7. The method of claim 1 wherein the training operation comprises:
training the machine learning model using metadata extracted from an entity knowledge graph.

8. The method of claim 1 wherein the generating operation comprises:
generating a different prediction of a different target performance result for the target entity for a different target time interval after a different target milestone in the time series by inputting to the trained machine learning model target query intents for the target entity allocated to a different time interval before the different target milestone.

9. A computing device having a processor and memory, the processor configured to execute instructions on the memory for predicting performance of a target entity, the computing device comprising:
a time-factored aggregator executable by the processor and configured to allocate training query intents generated from a query-URL click graph for multiple training entities into training time intervals in a time series based on a corresponding query intent time for each training query intent, the training time intervals in the time series being separated by training milestones in the time series, and to allocate training performance results for the multiple training entities into the training time intervals in the time series based on a corresponding performance time of each training performance result, wherein the training milestones represent occurrences of predefined types of temporally related events that influence performance of the training entities, wherein the time-factored aggregator is further configured to:
select one or more query intents from the training query intents allocated to a training time interval prior to a time of one of the training milestones; and
select one or more training performance results from the allocated training performance results after the time of the one of the training milestones; and

a machine learning model configured to receive the allocated training query intents and allocated training performance results, the machine learning model being trained for the one of the training milestones of the time series based on the selected one or more training query intents allocated to a training time interval prior to the time of one of the training milestones and the selected one or more training performance results allocated to a training time interval after the time of the one of the training milestones, the machine learning model being further configured to generate a prediction of a target performance result for the target entity for an interval after a target milestone in the time series by inputting to the trained machine learning model target query intents allocated to the target entity in a target time interval before the target milestone, wherein the target milestones represent occurrences of predefined types of temporally related events that influence performance of the target entities.

10. The computing device of claim 9 further comprising:
a query intent generator executable by the processor and configured to receive and

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Shrivastava Mayank
• Zhou Hui
• Shukla Pushpraj
• Kok Emre Hamit
• Mane Sonal Prakash
• Brisimitzis Dimitrios

Classifications

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F16/90335

  Query processing · CPC title

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title