Mining training data for training dependency model

Patent metadata

Field	Value
Publication number	US-11816636-B2
Application number	US-202117412753-A
Country	US
Kind code	B2
Filing date	Aug 26, 2021
Priority date	Aug 26, 2021
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

Techniques for mining training data for use in training a dependency model are disclosed herein. In some embodiments, a computer-implemented method comprises: obtaining training data comprising a plurality of reference skill pairs, each reference skill pair comprising a corresponding first reference skill and a corresponding second reference skill, the plurality of reference skill pairs being included in the training data based on a co-occurrence of the corresponding first and second reference skills for each reference skill pair in the plurality of reference skill pairs, the co-occurrence comprising the corresponding first and second reference skills co-occurring for a same entity; and training a dependency model with a machine learning algorithm using the training data, the dependency model comprising a logistic regression model or a data gradient boosted decision tree (GBDT) model. The dependency model may then be used to identify corresponding dependency relations for a plurality of target skill pairs.

First claim

What is claimed is:

1. A computer-implemented method performed by a computer system having a memory and at least one hardware processor, the computer-implemented method comprising:
obtaining training data comprising a plurality of reference skill pairs, each reference skill pair comprising a first reference skill and a second reference skill, the plurality of reference skill pairs being included in the training data based on the first and second reference skills for each reference skill pair in the plurality of reference skill pairs co-occurring for a same entity, the plurality of reference skill pairs being included in the training data is further based on an indication of a temporal order of the first and second reference skills for the same entity for each reference skill pair in the plurality of reference skill pairs;
training a dependency model with a first machine learning algorithm using the plurality of reference skill pairs of the training data, the plurality of reference skill pairs being used in the training of the dependency model based on the indication of the temporal order of the first and second reference skills for the same entity for each reference skill pair in the plurality of reference skill pairs, the dependency model comprising a logistic regression model or a data gradient boosted decision tree (GBDT) model;
training a Bidirectional Encoder Representations from Transformers (BERT) model with a second machine learning algorithm using the plurality of reference skill pairs of the training data, the plurality of reference skill pairs being used in the training of the BERT model based on the indication of the temporal order of the first and second reference skills for the same entity for each reference skill pair in the plurality of reference skill pairs;
identifying a dependency relation for each target skill pair in a plurality of target skill pairs using the dependency model, each target skill pair in the plurality of target skill pairs comprising a first target skill and a second target skill;
for each target skill pair in the plurality of target skill pairs, determining a relation direction using the BERT model; and
using the identified dependency relation and determined relation direction for at least a portion of the plurality of target skill pairs in an application of an online service.

2. The computer-implemented method of claim 1 , wherein the indication of the temporal order comprises an indication of a chronological sequence that includes the first and second reference skills for the same entity.

3. The computer-implemented method of claim 1 , wherein the same entity comprises a sequential playlist of online courses available to users for viewing via the online service, the plurality of reference skill pairs being included in the training data is further based on the first reference skill being associated with a first online course in the sequence of online courses and the second reference skill being associated with a second online course in the sequence of online courses.

4. The computer-implemented method of claim 1 , wherein the same entity comprises a history of online courses that have been watched by a user of the online service, the history of online courses being stored in association with the user, the plurality of reference skill pairs being included in the training data is further based on the first reference skill being associated with a first online course in the history of online courses and the second reference skill being associated with a second online course in the history of online courses.

5. The computer-implemented method of claim 1 , wherein the same entity comprises a history of a user adding skills to a profile of the user, the profile being stored in a database of the online service.

6. The computer-implemented method of claim 1 , wherein the same entity comprises an online job posting that includes the first reference skill and the second reference skill.

7. The computer-implemented method of claim 1 , further comprising generating a directed graph using the identified dependency relation and determined relation direction for the plurality of target skill pairs, wherein the using the at least a portion of the plurality of target skills in the application of the online service comprises using the directed graph in the application of the online service.

8. The computer-implemented method of claim 1 , wherein the using the identified dependency relation and determined relation direction for at least a portion of the plurality of target skill pairs in the application of the online service comprises:
displaying, on a computing device of a target user, a selectable user interface element for each second target skill in the at least a portion of the plurality of target skill pairs, the selectable user interface element being configured to trigger storing of the second target skill as part of the profile of the target user in response to a selection of the selectable user interface element.

9. The computer-implemented method of claim 1 , wherein the using the identified dependency relation and determined relation direction for at least a portion of the plurality of target skill pairs in the application of the online service comprises:
displaying, on a computing device of a target user, a selectable user interface element for an online job posting, the selectable user interface element being configured to, in response to its selection, trigger a display of the online job posting on the computing device of the target user or initiate an online application process for the online job posting on the computing device of the target user.

10. The computer-implemented method of claim 1 , wherein the using the identified dependency relation and determined relation direction for at least a portion of the plurality of target skill pairs in the application of the online service comprises:
determining that a profile of a target user of the online service includes the first target skill of each target skill pair in the at least a portion of the plurality of target skill pairs;
determining that an online course includes the second target skill of each target skill pair in the at least a portion of the plurality of target skill pairs;
selecting the online course based on the determining that the profile of the target user includes the first target skill of each target skill pair in the at least a portion of the plurality of target skill pairs, the determining that the online course includes the second target skill of each target skill pair in the at least a portion of the plurality of target skill pairs, and the identified dependency relation and determined relation direction for the at least a portion of the plurality of target skill pairs; and
displaying, on a computing device of the target user, a selectable user interface element for the online course based on the selecting the online course, the selectable user interface element being configured to, in response to its selection, trigger a playing of a multimedia file of the online course on the computing device or initiate an online process for playing the multimedia file of the online course on the computing device.

11. The computer-implemented method of claim 1 , wherein the using the identified dependency relation and determined relation direction for at least a portion of the plurality of target skill pairs in the application of the online service comprises:
determining that a profile of a first target user of the online service includes the first target skill of each target skill pair in the at least a portion of the plurality of target skill pairs;
determining that a search query submitted by a second t

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wu Liwei
• Ma Wenjia
• Yang Jaewon
• Li Yanen

Classifications

• G06Q10/1053 Primary

  Employment or hiring · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N20/20

  Ensemble learning · CPC title

• G06Q10/063112

  Skill-based matching of a person or a group to a task · CPC title

• G06N3/045

  Combinations of networks · CPC title