Generation of training data for ideal candidate search ranking model

Patent metadata

Field	Value
Publication number	US-10606847-B2
Application number	US-201615169346-A
Country	US
Kind code	B2
Filing date	May 31, 2016
Priority date	May 31, 2016
Publication date	Mar 31, 2020
Grant date	Mar 31, 2020

Abstract

In an example embodiment, one or more sample ideal candidate member profiles in a social networking service are obtained, as well as one or more sample search result member profiles in the social networking service. Then, for each unique pair of sample ideal candidate member profile and sample search result member profile, a label is generated using a score generated from log information of the social networking service, the log information including records of communications between a searcher and members of the social networking service, the score being higher if the searcher communicated with both the member corresponding sample ideal candidate member profile and the member corresponding to the sample search result member profile in a same search session. The generated labels are fed into a machine learning algorithm to train a combined ranking model used to output ranking scores for search result member profiles.

First claim

What is claimed is:

1. A computer implemented method, comprising:
obtaining one or more sample ideal candidate member profiles in a social networking service;
obtaining one or more sample search result member profiles in the social networking service, the one or more sample search result member profiles being results returned in response to searches performed using the one or more sample ideal candidate member profiles as input;
identifying one or more pairs of sample ideal candidate member profiles and sample search result member profile, wherein each pair includes a sample ideal candidate member profile and a sample search result member profile returned in response to a search performed using the corresponding sample ideal candidate member profile in the pair as input;
generating, for each unique pair of sample ideal candidate member profile and sample search result member profile, a label using a score generated from log information of the social networking service, the log information including records of communications between a searcher and members of the social networking service, the score being different if the searcher communicated with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in a same search session than if the searcher did not communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same search session, the label being indicative of the likelihood that the searcher would interact with the sample search result member profile in the pair in response to the search being performed using the corresponding sample ideal candidate member profile as input; and
feeding the generated labels into a machine learning algorithm to train a combined ranking model used to output ranking scores for search result member profiles.

2. The method of claim 1 , wherein the score is a medium score between the score if the searcher did communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same session and the score if the searcher did not communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same session if the log information further includes records of user input by the searcher, the user input causing interaction with member profiles in the social networking service but not resulting in communications between the searcher and the member of the social networking service corresponding to both the sample ideal candidate member profile and the sample search result member profile in the same search session.

3. The method of claim 1 , wherein a search session is a browsing session.

4. The method of claim 1 , wherein a search session includes a period of time between a searcher initiating a search and the searcher submitting an unrelated search or logging off the social networking service.

5. The method of claim 2 , wherein the score if the searcher communicated with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in a same search session is higher than the score if the searcher did not communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same search session.

6. The method of claim 1 , further comprising aggregating label scores from multiple search sessions.

7. The method of claim 1 , further comprising ranking one or more search result member profiles using the combined ranking model.

8. A system comprising:
a hardware processor; and
a computer-readable medium having instructions stored thereon, which, when executed by the processor, cause the system to:
obtain one or more sample ideal candidate member profiles in a social networking service;
obtain one or more sample search result member profiles in the social networking service, the one or more sample search result member profiles being results returned in response to searches performed using the one or more sample ideal candidate member profiles as input;
identify one or more pairs of sample ideal candidate member profiles and sample search result member profile, wherein each pair includes a sample ideal candidate member profile and a sample search result member profile returned in response to a search performed using the corresponding sample ideal candidate member profile in the pair as input;
generate, for each unique pair of sample ideal candidate member profile and sample search result member profile, a label using a score generated from log information of the social networking service, the log information including records of communications between a searcher and members of the social networking service, the score being different if the searcher communicated with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in a same search session than if the searcher did not communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same search session, the label being indicative of the likelihood that the searcher would interact with the sample search result member profile in the pair in response to the search being performed using the corresponding sample ideal candidate member profile as input; and
feed the generated labels into a machine learning algorithm to train a combined ranking model used to output ranking scores for search result member profiles.

9. The system of claim 8 , wherein the score is a medium score between the score if the searcher did communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same session and the score if the searcher did not communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same session if the log information further includes records of user input by the searcher, the user input causing interaction with member profiles in the social networking service but not resulting in communications between the searcher and the member of the social networking service corresponding to both the sample ideal candidate member profile and the sample search result member profile in the same search session.

10. The system of claim 8 , wherein a search session is a browsing session.

11. The system of claim 8 , wherein a search session includes a period of time between a searcher initiating a search and the searcher submitting an unrelated search or logging off the social networking service.

12. The system of claim 9 , wherein the score if the searcher communicated with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in a same search session is higher than the score if the searcher did not communicate with both the member corresponding to the sample ideal candidate member profile and the member corresponding to the sample search result member profile in the same search session.

13. The system of claim 8 , wherein the instruc

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Yan Yan
• Thuc Ha Viet
• Wu Xianren
• Kanduri Satya Pradeep
• Dialani Vijay
• Xu Ye
• Gupta Abhishek
• Sinha Shakti Dhirendraji

Classifications

• G06N3/08

  Learning methods · CPC title

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title

• G06F16/24578 Primary

  using ranking · CPC title

• G06N20/00

  Machine learning · CPC title

• G06Q50/01

  Physics · mapped topic