Ranking user comments on media using reinforcement learning optimizing for session dwell time

Patent metadata

Field	Value
Publication number	US-12340311-B2
Application number	US-202318132499-A
Country	US
Kind code	B2
Filing date	Apr 10, 2023
Priority date	Jun 19, 2019
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

A method is provided, including: storing comments generated in response to a content item served over a network; analyzing the comments to determine features associated with each of the comments; using a scoring model to score each comment based on the comment's corresponding features; receiving a request to serve a subset of the comments; responsive to the request, selecting a ranking of the comments that is one permutation from possible rankings of the comments, wherein selecting the ranking is in accordance with a probability distribution of the possible rankings that is based on the scores of the comments; serving comments identified by the selected ranking over the network to a client device; determining a dwell time on the served comments; applying the dwell time to update the scoring model.

First claim

What is claimed is:

1. A method, comprising:
using a scoring model to score each comment, of comments, based on corresponding features of the comment;
receiving, from a client device, a request to serve a subset of the comments comprising some but not all of the comments scored using the scoring model;
determining a plurality of possible rankings of the comments associated with a plurality of possible permutations, wherein the plurality of possible rankings of the comments comprises a first possible ranking of the comments associated with a first possible permutation and a second possible ranking of the comments associated with a second possible permutation;
responsive to the request to serve the subset of the comments comprising some but not all of the comments scored using the scoring model, selecting a ranking of the comments that is one permutation from the plurality of possible rankings of the comments, wherein selecting the ranking is in accordance with a probability distribution of the plurality of possible rankings that is based on scores of the comments, wherein the ranking of the comments is associated with increasing a reward at a given time by representing the comments in an order of the ranking, receiving one or more measurable reactions as a scalar reward for the ranking, and updating a ranking mechanism to increase the reward; and
serving one or more comments identified by the selected ranking over a network to the client device.

2. The method of claim 1 , wherein each permutation from the plurality of possible rankings defines a unique order for at least some of the comments.

3. The method of claim 1 , wherein the features include statistics based on at least one of replies, upvotes, downvotes, or age.

4. The method of claim 1 , wherein the scoring model includes a neural network.

5. The method of claim 4 , wherein updating the scoring model includes adjusting one or more weights of the neural network.

6. The method of claim 1 , wherein updating the scoring model is configured to adjust the scoring model to maximize one or more dwell times of comments ordered according to one or more scores as determined by the scoring model.

7. The method of claim 1 , wherein the scoring model is configured to provide for a given comment a score indicating a relative contribution of the given comment to an optimized ranking.

8. The method of claim 1 , wherein the probability distribution is defined from probabilities of permutations of the plurality of possible rankings, such that a given ranking prioritizing comments predicted to have greater dwell time has a higher probability than a given ranking prioritizing comments predicted to have a lower dwell time, as determined from the scoring of each comment.

9. The method of claim 1 , comprising:
determining a value based on the serving the one or more comments; and
updating the scoring model based on the value.

10. The method of claim 1 , wherein the request is generated from an access to a content item.

11. The method of claim 1 , wherein selecting the ranking includes sequentially defining a multinomial distribution and sampling from the multinomial distribution.

12. The method of claim 9 , wherein determining the value includes tracking a quantity of time that the one or more comments are presented at the client device.

13. A non-transitory computer readable medium having program instructions embodied thereon, the program instructions being configured, when executed by a computing device, to cause the computing device to perform operations comprising:
using a scoring model to score each comment, of comments, based on corresponding features of the comment;
receiving a request to serve a subset of the comments;
determining a plurality of possible rankings of the comments associated with a plurality of possible permutations;
responsive to the request, selecting a ranking of the comments that is one permutation from the plurality of possible rankings of the comments, wherein selecting the ranking is in accordance with a probability distribution of the plurality of possible rankings that is based on scores of the comments, wherein the ranking of the comments is associated with increasing a reward at a given time by representing the comments in an order of the ranking, receiving one or more measurable reactions as a scalar reward for the ranking, and updating a ranking mechanism to increase the reward; and
serving one or more comments identified by the selected ranking over a network to a client device,
wherein at least one of:
the operations comprise adjusting the scoring model to maximize one or more dwell times of comments ordered according to one or more scores; or
the probability distribution is defined from probabilities of permutations of the plurality of possible rankings, such that a given ranking prioritizing comments predicted to have greater dwell time has a higher probability than a given ranking prioritizing comments predicted to have a lower dwell time.

14. The non-transitory computer readable medium of claim 13 , wherein each permutation from the plurality of possible rankings defines a unique order for at least some of the comments.

15. The non-transitory computer readable medium of claim 13 , wherein the features include statistics based on at least one of replies, upvotes, downvotes, or age.

16. The non-transitory computer readable medium of claim 13 , wherein the scoring model includes a neural network.

17. The non-transitory computer readable medium of claim 16 , wherein updating the scoring model includes adjusting one or more weights of the neural network.

18. The non-transitory computer readable medium of claim 13 , wherein the plurality of possible rankings of the comments comprises a first possible ranking of the comments associated with a first possible permutation and a second possible ranking of the comments associated with a second possible permutation.

19. The non-transitory computer readable medium of claim 13 , wherein the scoring model is configured to provide for a given comment a score indicating a relative contribution of the given comment to an optimized ranking.

20. A system comprising at least one server computer, the at least one server computer having:
logic for using a scoring model to score each comment, of comments, based on corresponding features of the comment, wherein an objective associated with the scoring model is changed without re-training the scoring model;
logic for receiving a request to serve a subset of the comments;
logic for determining a plurality of possible rankings of the comments associated with a plurality of possible permutations;
logic for, responsive to the request, selecting a ranking of the comments that is one permutation from the plurality of possible rankings of the comments, wherein selecting the ranking is in accordance with a probability distribution of the plurality of possible rankings that is based on scores of the comments, wherein the ranking of the comments is associated with increasing a reward at a given time by representing the comments in an order of the ranking, receiving one or more measurable reactions as a scalar reward for the ranking, and updating a ranking mechanism to increase the reward; and
logic for serving one or more comments identified by the selected ranking over a network to a client device.

Assignees

• Yahoo Assets Llc

Inventors

• Thadani Kapil
• Soni Akshay
• Shah Parikshit
• Chevalier Troy
• Ramakrishnan Sreekanth
• Nagao Aaron
• Qu Zhi

Classifications

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/006 Primary

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06F16/338

  Presentation of query results · CPC title

• G06N3/02

  Neural networks · CPC title

• G06N7/026

  Development tools for entering the parameters of a fuzzy system · CPC title