Systems and methods for estimating user judgment based on partial feedback and applying it to message categorization

Patent metadata

Field	Value
Publication number	US-2016156579-A1
Application number	US-201414557307-A
Country	US
Kind code	A1
Filing date	Dec 1, 2014
Priority date	Dec 1, 2014
Publication date	Jun 2, 2016
Grant date	—

Abstract

Messages in a first and second plurality of messages are respectively classified using a first and second classifier into message categories in a set of message categories, with messages in the first and second plurality of messages being associated with message reputation carriers in a plurality of message reputation carriers. The classified messages are delivered to recipients and message category correction events are collected. Correction weights are determined for correction types associated with the set of message categories using the initial message categorizations and the category correction events. At least a subset of the calculated correction weights is used to determine a probability or likelihood that a particular message reputation carrier in the plurality of carriers is associated with a first message category in the set of message categories. The particular carrier is whitelisted to the first message category when the calculated probability or likelihood satisfies a whitelisting criterion.

First claim

What is claimed is:

1 . A method of whitelisting a first message reputation carrier to a first message category in a set of message categories, the method comprising:
at a computer system having one or more processors, and memory storing one or more programs for execution by the one or more processors:
classifying each message in a first plurality of messages using a first classifier, thereby independently identifying an initial message category in the set of message categories for each respective message in the first plurality of messages, wherein the first plurality of messages includes, for each respective message reputation carrier in a plurality of message reputation carriers, at least one message associated with the respective message reputation carrier and wherein the plurality of message reputation carriers includes the first message reputation carrier;
classifying each message in a second plurality of messages using a second classifier, thereby independently identifying an initial message category in the set of message categories for each respective message in the second plurality of messages, wherein the second plurality of messages includes, for each respective message reputation carrier in the plurality of message reputation carriers, at least one message associated with the respective message reputation carrier;
delivering the first and second plurality of messages to a plurality of recipients with a designation of the message category of each respective message in the first and second plurality of messages, as respectively determined by the first and second classifier;
collecting a plurality of recipient initiated message category correction events for messages in the first and second plurality of messages;
determining a correction weight for each respective correction type associated with the set of message categories using at least (i) the initial message category for each respective message in the first plurality of messages assigned by the first classifier, (ii) the initial message category for each respective message in the second plurality of messages assigned by the second classifier and (iii) the plurality of recipient initiated message category correction events;
using the correction weight for each correction type associated with a message category in the set of message categories to determine a probability or likelihood that the first message reputation carrier is associated with the first message category in the set of message categories; and
whitelisting the first message reputation carrier to the first message category when the calculated probability or likelihood satisfies a whitelisting criterion.

2 . The method of claim 1 , wherein the set of message categories comprises promotions, social, updates, and forums.

3 . The method of claim 1 , wherein
the first classifier and the second classifier are the same classifier, the first plurality of messages is classified by the classifier at a time before a subset of message reputation carriers in the plurality of message reputation carriers are whitelisted to message categories in the set of message categories, and the second plurality of messages is classified by the classifier at a time after the subset of message reputation carriers in the plurality of message reputation carriers are whitelisted to message categories.

4 . The method of claim 1 , wherein the determining a correction weight for each respective correction type associated with the set of message categories comprises minimizing the loss function:

loss
=

∑

i
,

j
≠
i

,
m





(

p

i
,
j
,
m

-

w

i
,
j

·

c

i
,
j
,
m

/

N
m

)

2

+

∑

i
,
m





(

∑
j





p

i
,
j
,
m

-

c

i
,
j
,
m

/

N
m

)

2

+

∑

j
,
k





(

∑
i

Assignees

• Google Inc

Inventors

• Kaufmann Tobias

Classifications

• H04L51/12

  Electricity · mapped topic

• H04L51/22 Primary

  Electricity · mapped topic

• H04L51/42 Primary

  Mailbox-related aspects, e.g. synchronisation of mailboxes · CPC title

• H04L51/212

  using filtering or selective blocking · CPC title