Systems and methods for selecting third party content based on feedback

Patent metadata

Field	Value
Publication number	US-10223742-B2
Application number	US-201514836537-A
Country	US
Kind code	B2
Filing date	Aug 26, 2015
Priority date	Aug 26, 2015
Publication date	Mar 5, 2019
Grant date	Mar 5, 2019

Abstract

The present disclosure selects third party content based on feedback. A selector identifies several content items including first and second content items (or more) responsive to a request. A machine learning engine determines a first feature of the first content item, a second feature of the second content item, and a third feature of the web page or a device associated with the request. The machine learning engine determines, responsive to the first feature and the third feature, a first score for the first content item based on a machine learning model generated using historical signals received from devices via a metadata channel formed from an electronic feedback interface. The machine learning engine determines a second score for the second content item responsive to the second feature and the third feature. A bidding module determines a price for the first content item based on the first and second scores.

First claim

What is claimed is:

1. A method of selecting content using electronic content selection infrastructure, comprising:
receiving, by a content selector executed by a data processing system, via a computer network, a request for content for display with a web page on a computing device;
identifying, by the content selector via an online content item selection process, a first candidate content item and a second candidate content item responsive to the request;
determining, by a machine learning engine executed by the data processing system, a first feature of the first candidate content item, a second feature of the second candidate content item, and a third feature corresponding to at least one of the web page and the computing device associated with the request;
generating, by the machine learning engine, a first predictor of likelihood of interaction with the first candidate content item based on a first combination of the first feature and the third feature, and a second predictor of likelihood of interaction with the second candidate content item based on a second combination of the second feature and the third feature;
determining, by the machine learning engine responsive to the first predictor of likelihood of interaction generated based on the first combination, a first score corresponding to likelihood of interaction for the first candidate content item based on a machine learning model generated using historical signals received from a plurality of computing devices via a metadata channel formed from an electronic feedback interface;
determining, by the machine learning engine responsive to the second predictor of likelihood of interaction generated based on the second combination, a second score corresponding to likelihood of interaction for the second candidate content item based on the machine learning module;
selecting, by the content selector for display with the web page on the computing device, the first candidate content item as a selected content item based on a comparison of the first score and the second score;
adjusting, by the data processing system, a parameter for the selected content item based on a difference between the first score of the first candidate content item and the second score of the second candidate content item; and
providing, by the data processing system, the adjusted parameter for input into the online content item selection process to cause the online content item selection process to select the selected content item for display with the web page on the computing device.

2. The method of claim 1 , further comprising:
providing, by the data processing system to the computing device, an instance of the electronic feedback interface responsive to displaying the selected content item;
receiving, by the data processing system via a metadata channel formed from the instance of the electronic feedback interface, a signal, the signal input via the computing device; and
updating, by the machine learning engine, the machine learning model based on the signal, the first feature and the third feature.

3. The method of claim 1 , further comprising:
providing, by the data processing system, an instance of the electronic feedback interface at least partially overlaid on the selected content item displayed with the web page on the computing device, wherein the instance of the electronic feedback interface includes at least one of an input text box, input button, or input drop down menu.

4. The method of claim 1 , further comprising:
receiving, by the data processing system from the plurality of computing devices, the historical signals in response to instances of the electronic feedback interface provided with previously displayed content items; and
generating, by the machine learning engine, the machine learning model using the historical signals and corresponding features.

5. The method of claim 1 , further comprising:
determining, by the data processing system, an initial ranking of the first candidate content item and the second candidate content item in an online content item auction initiated in response to the request, the first candidate content item ranking lower than the second candidate content item in the initial ranking; and
increasing, by the data processing system, a rank of the first candidate content item based on the first score and the second score, wherein the first candidate content item is a highest ranking content item in the online content item auction.

6. The method of claim 1 , further comprising:
generating, by the machine learning engine, the machine learning model using at least one of a neural network, linear regression technique, or a Bayesian estimator.

7. The method of claim 1 , wherein at least one of the historical signals includes a dislike signal comprising a binary value.

8. The method of claim 1 , wherein the first feature includes at least one of a keyword, a topic, a content provider, a content selection criterion, or a content vertical.

9. The method of claim 1 , wherein the third feature includes at least one of a keyword of the web page, a topic of the web page, a domain name of the web page, a location of the computing device, a profile associated with the computing device, or historical browsing activity associated with the computing device.

10. The method of claim 1 , further comprising:
generating the first predictor of likelihood of interaction with the first candidate content item based on a keyword of the first candidate content item and a topic of the web page.

11. A system for selection of content using electronic content selection infrastructure, comprising:
a data processing system comprising one or more processors and memory;
a content selector executed by the data processing system to receive, via a computer network, a request for content for display with a web page on a computing device;
the content selector identifies, via an online content item selection process, a first candidate content item and a second candidate content item responsive to the request;
a machine learning engine executed by the data processing system to determine a first feature of the first candidate content item, a second feature of the second candidate content item, and a third feature corresponding to at least one of the web page and the computing device associated with the request;
the machine learning engine generates a first predictor of likelihood of interaction with the first candidate content item based on a first combination of the first feature and the third feature, and a second predictor of likelihood of interaction with the second candidate content item based on a second combination of the second feature and the third feature;
the machine learning engine determines, responsive to the first predictor of likelihood of interaction generated based on the first combination, a first score corresponding to likelihood of interaction for the first candidate content item based on a machine learning model generated using historical signals received from a plurality of computing devices via a metadata channel formed from an electronic survey interface;
the machine learning engine determines, responsive to the second predictor of likelihood of interaction generated based on the second combination, a second score corresponding to likelihood of interaction for the second candidate content item based on the machine learning module;
the content selector selects, for display with the web page on the computing device, the first candidate content item based on a comparison of the first score and the second score;
the data processing system to:
adjust a parameter for the first candidate content item based on a difference

Assignees

• Google Inc
• Google Llc

Inventors

• Arora Shilpa
• Mcculloch Colin
• Yagnik Niyati
• Thomas Creighton
• Prabhu Manohar
• Lipus Timothy
• Aiello Michael Eugene
• Zhang Yi
• Bangla Ajay Kumar
• Rabii Bahman
• Zhao Gaofeng
• Cui Yingweii

Classifications

• G06Q30/08 Primary

  Auctions · CPC title

• G06F17/3053

  Physics · mapped topic

• G06F17/3089

  Physics · mapped topic

• G06F17/30864

  Physics · mapped topic

• G06N99/005

  Physics · mapped topic