Adjusting demand for order fulfillment during various time intervals for order fulfillment by an online concierge system

Patent metadata

Field	Value
Publication number	US-12175487-B2
Application number	US-202318503078-A
Country	US
Kind code	B2
Filing date	Nov 6, 2023
Priority date	Jul 29, 2021
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

An online concierge system allows users to order items within discrete time intervals later than a time when an order was received or for short-term fulfillment when the order was received. To account for a number of shoppers available to fulfill orders during different discrete time intervals and numbers of orders for fulfillment during different discrete time intervals, the online concierge system specifies a target rate for orders fulfilled later than a specified discrete time interval and a threshold from the target rate. A trained machine learning model periodically predicts a percentage of orders being fulfilled late, with an order associated with a predicted percentage when the order was received. The online concierge system increases a price of orders associated with predicted percentages greater than the threshold from the target rate. The increased price of an order is determined from a price elasticity curve and the predicted percentage.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
at an online concierge system comprising at least one processor and memory:
maintaining a plurality of time intervals for fulfilling orders received by the online concierge system;
receiving a plurality of orders for fulfillment;
assigning forecasted number of orders for each time interval of the plurality of time intervals;
training a machine learning model based on a set of training data, the machine learning model configured to predict a conversion rate within a time interval, wherein training the machine learning model comprises:
applying the machine learning model to samples of the training data, each sample with a label indicating whether the sample was fulfilled within a target time interval or was fulfilled after the target time interval, and
modifying one or more parameters of the machine learning model until a difference between a predicted percentage of fulfillment outputted by the machine learning model and recorded percentage of fulfillment reflected by labels in the training data satisfies one or more criteria;

retraining the machine learning model, wherein retraining the machine learning model comprises:
determining that the predicted percentage of fulfillment after the specific time interval for the requested order is less than a threshold,
responsive to determining that the predicted percentage of fulfillment after the specific time interval for the requested order is not less than the threshold, determining an increased rate for fulfillment of the requested order during the specific time interval, the increased rate greater than a standard rate for the specific time interval, and
updating the machine learned model following retraining of the machine learned model with new training datasets;

generating, for each time interval of the plurality of time intervals, an elasticity curve based on the conversion rate predicted by the machine learning model;
receiving a request from a user for a requested order at the online concierge system, the request identifying a specific time interval for fulfillment of the requested order;
retrieving a specific elasticity curve generated for the specific time interval;
selecting a conversion rate for the specific time interval; and
transmitting, for display at an interface of a client device of the user including, a specific rate for the specific time interval determined based on the conversion rate and the specific elasticity curve generated for the specific time interval.

2. The computer-implemented method of claim 1 , wherein the specific rate is determined from a rate at which users place orders with a standard price, a number of received requests for orders having a predicted percentage of being fulfilled after the specific time interval that is greater than a threshold from the target rate and less than the target rate, and a number of received requests for orders with a predicted percentage of being fulfilled after the specific time interval greater than the target rate.

3. The computer-implemented method of claim 1 , further comprising:
identifying an additional time interval fulfilling the requested order later than the specific time;
obtaining a demand pacing curve for the additional time interval, the demand pacing curve specifying a rate at which users place orders with the online concierge system identifying the additional time interval at different times so a number of orders identifying the additional time interval equals a forecasted value for the additional time interval at a start of the additional time interval;
determining a current number of orders received by the online concierge system identifying the additional time interval for fulfillment at the time when the request was received;
determining that the current number of orders is less than a value for the time when the request was received from the demand pacing curve for the additional time interval;
responsive to determining the current number of orders is not less than the value for the time when the request was received from the demand pacing curve for the additional time interval, generating an increased price for the additional time interval; and
transmitting the interface from the online concierge system to the client device of the user including the increased price determined for the additional time interval for display to the user.

4. The computer-implemented method of claim 1 , wherein generating, for each time interval of the plurality of time intervals, the elasticity curve based on the conversion rate predicted by the machine learning model comprises:
retrieving historical orders received from users and rates presented to the users;
determining changes in prices for fulfilling an order affects a rate at which the users place orders with the online concierge system for fulfillment; and
determining the elasticity curve according to the changes in prices relative to the rate at which the users place orders with the online concierge system for fulfillment.

5. The computer-implemented method of claim 1 , wherein generating, for each time interval of the plurality of time intervals, the elasticity curve based on the conversion rate predicted by the machine learning model comprises:
retrieving historical orders received from users and rates presented to the users;
applying a regression analysis to analyze the historical orders relative to the rates presented to the users; and
determining the elasticity curve according to the regression analysis.

6. The computer-implemented method of claim 1 , wherein generating, for each time interval of the plurality of time intervals, the elasticity curve based on the conversion rate predicted by the machine learning model comprises:
retrieving historical orders received from users and rates presented to the users;
applying an interpolation to analyze the historical orders relative to the rates presented to the users; and
determining the elasticity curve according to the interpolation.

7. The computer-implemented method of claim 1 , wherein selecting the conversion rate for the specific time interval comprises:
generating a demand curve for the specific time interval, the demand curve identifying a percentage of orders that are received before the specific time interval;
generating a demand pacing curve specifying a rate at which users place orders with the online concierge system; and
selecting the conversion rate for the specific time interval based on the demand pacing curve.

8. A non-transitory computer-readable medium configured to store code comprising instructions, wherein the instructions, when executed by one or more processors, cause the one or more processors to:
maintain a plurality of time intervals for fulfilling orders received by an online concierge system;
receive a plurality of orders for fulfillment;
assign forecasted number of orders for each time interval of the plurality of time intervals;
train a machine learning model based on a set of training data, the machine learning model configured to predict a conversion rate within a time interval, wherein training the machine learning model comprises:
applying the machine learning model to samples of the training data, each sample with a label indicating whether the sample was fulfilled within a target time interval or was fulfilled after the target time interval, and
modifying one or more parameters of the machine learning model until a difference between a predicted percentage of fulfillment outputted by the machine learning model and recorded percentage of fulfillment reflected by labels in the training data satisfies one or more criteria;

retrain the machine learning model, wherein retraining the machine

Assignees

• Maplebear Inc

Inventors

• Deng Houtao
• Chen Ji
• Wang Zi
• Zeliger Soren
• Krishnan Ganesh
• Yuan Wa
• Scheibe Michael

Classifications

• G06Q10/06313

  Resource planning in a project environment · CPC title

• G06N20/00

  Machine learning · CPC title

• G06Q30/0205

  based on location or geographical consideration · CPC title

• G06Q10/06315

  Needs-based resource requirements planning or analysis · CPC title

• G06N5/04

  Inference or reasoning models · CPC title