Online active learning in user-generated content streams

Patent metadata

Field	Value
Publication number	US-2020137012-A1
Application number	US-201916729813-A
Country	US
Kind code	A1
Filing date	Dec 30, 2019
Priority date	Oct 26, 2011
Publication date	Apr 30, 2020
Grant date	—

Abstract

Software for online active learning receives content posted to an online stream at a website. The software converts the content into an elemental representation and inputs the elemental representation into a probit model to obtain a predictive probability that the content is abusive. The software also calculates an importance weight based on the elemental representation. And the software updates the probit model using the content, the importance weight, and an acquired label if a condition is met. The condition depends on an instrumental distribution. The software removes the content from the online stream if a condition is met. The condition depends on the predictive probability, if an acquired label is unavailable.

First claim

What is claimed is:

1 . A method, comprising:
receiving content comprising a plurality of user generated content (UGC) items, wherein at least a portion of the plurality of UGC items is abusive; processing the plurality of UGC items using a predictive model to obtain a predictive probability of whether one or more UGC items of the plurality of UGC items is abusive or not abusive; updating, in response to receipt of one or more labels, the predictive model using the one or more labels; applying the predictive model to the plurality of UGC items to identify one or more abusive UGC items; and serving, to a client device, second content comprising a portion of the content but not comprising the one or more abusive UGC items.

2 . The method of claim 1 , wherein the predictive model includes a memory loss factor.

3 . The method of claim 2 , wherein the memory loss factor enables the predictive model to adapt to concept drift of UGC items.

4 . The method of claim 1 , wherein UGC items that are associated with a predictive probability of being abusive have a placement within the probabilistic queue that is ahead of UGC items that are associated with a predictive probability of being not abusive.

5 . The method of claim 1 , wherein the processing the plurality of UGC items is configured to obtain an importance weight for each of the plurality of UGC items, and wherein the updating the predictive model includes adjusting the importance weight using the one or more labels.

6 . The method of claim 1 , comprising selecting specific ones of the plurality of UGC items based on an instrumental distribution.

7 . The method of claim 1 , wherein the plurality of UGC items include text, wherein the text is processed as an elemental representation.

8 . The method of claim 1 , wherein the plurality of UGC items include an image, wherein the image is processed using a bag of features model.

9 . The method of claim 1 , wherein the portion of the plurality of UGC items that is abusive includes at least one of spam, fraudulent offers, illegal offers, offensive language, threatening language, or treasonous language.

10 . A computer-implemented method, comprising:
receiving a plurality of user generated content (UGC) items that has been posted to an online stream at a website, wherein at least of portion of the plurality of UGC items is abusive; processing the plurality of UGC items using a predictive model to obtain a predictive probability of whether one or more UGC items of the plurality of UGC items is abusive or not abusive; identifying one or more abusive UGC items based on the predictive probability; and delivering, to a client device when the client device accesses the website, content comprising a second portion of the plurality of UGC items but not comprising the one or more abusive UGC items.

11 . The computer-implemented method of claim 10 , wherein the processing the plurality of UGC items is configured to obtain an importance weight for a first UGC item of the plurality of UGC items.

12 . The computer-implemented method of claim 11 , wherein the predictive model is updated by adjusting the importance weight using one or more labels.

13 . The computer-implemented method of claim 10 , wherein UGC items having the predictive probability of being abusive are placed ahead of UGC items having the predictive probability of being not abusive within the queue.

14 . The computer-implemented method of claim 10 , comprising selecting specific ones of the plurality of UGC items associated with a label based on an instrumental distribution.

15 . The computer-implemented method of claim 10 , wherein at least a portion of the plurality of UGC items includes text, wherein the text is processed as elemental representations.

16 . The computer-implemented method of claim 10 , wherein at least a portion of the plurality of UGC items includes an image, wherein the image is processed using a bag of features model.

17 . The computer-implemented method of claim 10 , wherein the portion of the plurality of UGC items that is abusive includes at least one of spam, fraudulent offers, illegal offers, offensive language, threatening language, or treasonous language.

18 . The computer-implemented method of claim 10 , wherein the predictive model includes a memory loss factor that enables the predictive model to adapt to concept drift of UGC items.

19 . A non-transitory computer-readable medium storing a computer program that when executed by a processor-based system, performs operations comprising:
receiving a plurality of user generated content (UGC) items that has been posted to an online stream at a website, wherein at least of portion of the plurality of UGC items is abusive; processing the plurality of UGC items using a predictive model to obtain a predictive probability of whether one or more UGC items of the plurality of UGC items is abusive or not abusive; identifying one or more abusive UGC items based on the predictive probability; and delivering, to a client device when the client device accesses the website, content comprising a second portion of the plurality of UGC items but not comprising the one or more abusive UGC items.

20 . The non-transitory computer-readable medium of claim 19 , wherein the predictive model includes a memory loss factor.

Assignees

• Oath Inc

Inventors

• Chu Wei
• Zinkevich Martin
• Li Lihong
• Thomas Achint Oommen
• Tseng Belle

Classifications

• G06F15/16

  Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs {(coordinating program control therefor G06F9/52; in regulating and control system G05B)} · CPC title

• G06Q50/20 Primary

  Education · CPC title

• G06F11/00

  Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) · CPC title

• G06N5/02

  Knowledge representation; Symbolic representation · CPC title

• G06N7/00

  Computing arrangements based on specific mathematical models · CPC title