Identification of sample data items for re-judging

Patent metadata

Field	Value
Publication number	US-8935258-B2
Application number	US-48425609-A
Country	US
Kind code	B2
Filing date	Jun 15, 2009
Priority date	Jun 15, 2009
Publication date	Jan 13, 2015
Grant date	Jan 13, 2015

Abstract

Described is a technology for identifying sample data items (e.g., documents corresponding to query-URL pairs) having the greatest likelihood of being mislabeled when previously judged, and selecting those data items for re-judging. In one aspect, lambda gradient scores (information associated with ranked sample data items that indicates a relative direction and how “strongly” to move each data item for lowering a ranking cost) are summed for pairs of sample data items to compute re-judgment scores for each of those sample data items. The re-judgment scores indicate a relative likelihood of mislabeling. Once the selected sample data items are re-judged, a new training set is available, whereby a new ranker may be trained.

First claim

What is claimed is:

1. In a computing environment, a method comprising, employing at least one processor to perform steps comprising:
obtaining lambda gradient scores for pairs of ranked sample data items;
using the lambda gradient scores to compute re-judgment scores for the ranked sample data items, wherein each lambda gradient score includes a ranking cost for a pair of ranked sample data items; and
selecting one or more ranked sample data items of the ranked sample data it…

Assignees

• Svore Krysta M
• Abib Elbio Renato Torres
• Burges Christopher J C
• Middha Bhuvan
• Microsoft Corp

Inventors

• Svore Krysta M
• Abib Elbio Renato Torres
• Burges Christopher J C
• Middha Bhuvan

Classifications

• G06F18/2148

  Physics · mapped topic

• G06F16/951 Primary

  Physics · mapped topic

• G06F17/30864 Primary

  Physics · mapped topic

• G06K9/6257

  Physics · mapped topic

• G06F16/9566 Primary

  Physics · mapped topic