Adjusting demand for order fulfillment during various time intervals for order fulfillment by an online concierge system

Patent metadata

Field	Value
Publication number	US-11830018-B2
Application number	US-202117389281-A
Country	US
Kind code	B2
Filing date	Jul 29, 2021
Priority date	Jul 29, 2021
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

An online concierge system allows users to order items within discrete time intervals later than a time when an order was received or for short-term fulfillment when the order was received. To account for a number of shoppers available to fulfill orders during different discrete time intervals and numbers of orders for fulfillment during different discrete time intervals, the online concierge system specifies a target rate for orders fulfilled later than a specified discrete time interval and a threshold from the target rate. A trained machine learning model periodically predicts a percentage of orders being fulfilled late, with an order associated with a predicted percentage when the order was received. The online concierge system increases a price of orders associated with predicted percentages greater than the threshold from the target rate. The increased price of an order is determined from a price elasticity curve and the predicted percentage.

First claim

What is claimed is:

1. A method comprising:
maintaining, at an online concierge system, a plurality of discrete time intervals for fulfilling orders received by the online concierge system, each discrete time interval having a standard price for fulfillment;
determining a predicted percentage of orders fulfilled after a discrete time interval identified by an order at periodic intervals, the predicted percentage of orders fulfilled after the discrete time interval from a location identified by the order;
specifying a threshold of orders fulfilled after the discrete time interval identified by the order, the threshold percentage less than a target rate specifying a maximum percentage of orders fulfilled after the discrete time interval identified by the order;
receiving a request from a user for a requested order at the online concierge system identifying a specific discrete time interval for short-term fulfillment of the order within a specific duration of a time when the request was received;
determining a predicted percentage of fulfillment after the specific discrete time interval for the requested order as a predicted percentage of orders fulfilled after the discrete time interval determined for a periodic interval including the time when the request was received by applying a model trained from iterative application of the model to examples of training data labeled with a label indicating whether an example was fulfilled within a discrete time interval identified by the example or was fulfilled after the discrete time interval identified by the example and modification of one or more parameters of the model until a difference between a predicted percentage for an example output by the model and the label applied to the model satisfies one or more criteria;
training a machine learned model on a set of training data, wherein the set of training data includes labeled samples of previously fulfilled orders, with a label applied to a sample indicating whether a previously fulfilled order was fulfilled within a discrete time interval or was fulfilled after the discrete time interval, wherein training the machine learned model comprises:
applying the machine learned model to samples of training data each with the label indicating whether the sample was fulfilled within a discrete time interval or was fulfilled after the discrete time interval, and
modifying one or more parameters of the machine learned model until a difference between a predicted percentage of fulfillment outputted by the machine learned model and recorded percentage of fulfillment reflected by the labels in the training data satisfies one or more criteria;

retraining the machine learned model at various intervals iteratively, wherein retraining the machine learned model comprises:
determining that the predicted percentage of fulfillment after the specific discrete time interval for the requested order is less than the threshold,
responsive to determining that the predicted percentage of fulfillment after the specific discrete time interval for the requested order is not less than the threshold, determining an increased price for fulfillment of the requested order during the specific discrete time interval, the increased price greater than the standard price for the specific discrete time interval, and
updating the machine learned model following retraining of the machine learned model with new training datasets; and

transmitting an interface from the online concierge system to a client device of the user including the increased price determined for the specific discrete time interval for display to the user.

2. The method of claim 1 , further comprising:
responsive to determining the predicted percentage of fulfillment after the specific discrete time interval for the requested order is less than the threshold, selecting the standard price for the specific discrete time interval; and
transmitting an interface from the online concierge system to a client device of the user including the standard price determined for the specific discrete time interval for display to the user.

3. The method of claim 1 , further comprising:
responsive to determining the predicted percentage fulfillment after the discrete time interval for the requested order equals or exceeds the target rate, transmitting the interface to the user indicating the specific discrete time interval is unavailable for selection for fulfillment of the requested order.

4. The method of claim 1 , wherein determining the increased price for fulfillment of the requested order during the specific discrete time interval comprises:
retrieving a price elasticity curve correlating a rate at which users place orders with the online concierge system with different prices; and
selecting a price corresponding to a specific rate at which users place orders with the online concierge system from the price elasticity curve.

5. The method of claim 4 , wherein the specific rate is determined from a rate at which users place orders with the standard price, a number of received requests for orders having a predicted percentage of being fulfilled after the specific discrete time interval that is greater than the threshold from the target rate and less than the target rate, and a number of received requests for orders with a predicted percentage of being fulfilled after the specific discrete time interval greater than the target rate.

6. The method of claim 5 , wherein the specific rate comprises a ratio of:
(1) a product of the rate at which users place orders with the standard price and the number of received requests for orders having the predicted percentage of being fulfilled after the specific discrete time interval that is greater than the threshold from the target rate and less than the target rate, to
(2) a sum of the number of received requests for orders having the predicted percentage of being fulfilled after the specific discrete time interval that is greater than the threshold from the target rate and less than the target rate and the number of received requests for orders with the predicted percentage of being fulfilled after the specific discrete time interval greater than the target rate.

7. The method of claim 1 , wherein determining the predicted percentage of orders fulfilled after a discrete time interval identified by the order at periodic intervals comprises:
identifying a geographic region including the location; and
applying a trained model to a rate at which the online concierge system receives orders including locations within the determined geographic region, a rate at which shoppers select orders including locations within the determined geographic region for fulfillment, a number of orders including locations within the determined geographic region received by the online concierge system and not selected for fulfillment by shoppers, a number of shoppers available during the time interval in the geographic region, and historical information about traffic or road conditions in the geographic region, the trained model generating the predicted percentage of orders fulfilled after the discrete time interval identified by the order.

8. The method of claim 1 , further comprising:
identifying an additional discrete time interval fulfilling the requested order later than the specific duration from the time when the request for the requested order was received;
obtaining a demand pacing curve for the additional discrete time interval, the demand pacing curve specifying a rate at which users place orders with the online concierge system identifying the additional discrete time interval at different times so a number of orders identifying the additional discrete time interval equals a forecasted value for the additiona

Assignees

• Maplebear Inc

Inventors

• Deng Houtao
• Chen Ji
• Wang Zi
• Zeliger Soren
• Krishnan Ganesh
• Yuan Wa
• Scheibe Michael

Classifications

• G06Q30/0206 Primary

  Price or cost determination based on market factors · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06N20/00

  Machine learning · CPC title

• G06Q10/06312 Primary

  Adjustment or analysis of established resource schedule, e.g. resource or task levelling, or dynamic rescheduling · CPC title

• G06Q10/06313

  Resource planning in a project environment · CPC title