Replacing an unavailable item in an order using a trained outcome prediction model

Patent metadata

Field	Value
Publication number	US-12475503-B2
Application number	US-202318485797-A
Country	US
Kind code	B2
Filing date	Oct 12, 2023
Priority date	Oct 12, 2023
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

An online system receives orders from users and dispatches pickers to fulfill the orders by obtaining ordered items at a retailer. If an ordered item cannot be found by a picker, the picker may refund the item or attempt to find a replacement item. While obtaining a replacement item may increase revenue to the online system, it can also cause a bad outcome for user experience (e.g., an unacceptable replacement item, a refund request of the replacement item, etc.). To balance these interests, the online system trains a model to predict an outcome metric comprising a likelihood of a bad outcome from replacing an item or an expected amount of profit to the online system from a replacement item. The online system compares the outcome metric to a threshold to determine whether to promote or dissuade the picker from replacing a not-found item.

First claim

What is claimed is:

1 . A method, performed at a computer system comprising a processor and a non-transitory computer readable medium, comprising:
receiving an order from a user of the computer system, the order identifying one or more items and a retailer from which the one or more items are to be obtained; receiving, at the computer system, an indication from a picker that an item included in the order is unavailable at the retailer; determining a predicted outcome metric for replacing the item with another item by applying an outcome prediction model to the item, order features for the order, and user features of the user, the outcome prediction model trained by:
obtaining a training dataset including a plurality of training examples based on previously fulfilled orders, each training example including an item from a previously fulfilled order, a replacement item for the item from the previously fulfilled order, user features for a user from whom the previously fulfilled order was received, and order features for the previously fulfilled order, each training example having a label indicating an outcome of the previously fulfilled order;
applying the outcome prediction model to each training example of the training dataset to generate a predicted outcome metric for the training example when the item from the previously fulfilled order is replaced with the replacement item for the item from the previously fulfilled order;
scoring the outcome prediction model using a loss function based on the predicted outcome metric for the training example and the label of the training example; and
updating one or more parameters of the outcome prediction model by backpropagation based on the scoring until one or more criteria are satisfied;
determining that the predicted outcome metric for the order satisfies one or more criteria; and transmitting one or more instructions to a picker client device of the picker that modify one or more collection interfaces to increase a difficulty of identifying a replacement item for the item in response to determining that the predicted outcome metric for the order satisfies one or more criteria.

2 . The method of claim 1 , wherein the predicted outcome metric comprises a probability of a negative outcome from replacing the item of the order with the replacement item.

3 . The method of claim 2 , wherein the negative outcome comprises at least one of: the computer system receiving negative feedback for the replacement item when included in the order from the user, the computer system receiving an appeasement request from the user when the replacement item is included in the order, or the computer system receiving an indication the item was missing from the order from the user.

4 . The method of claim 2 , wherein transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining the predicted outcome metric for the order satisfies one or more criteria comprises:
transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining the predicted probability of the negative outcome from replacing the item of the order with the replacement item is greater than a threshold probability.

5 . The method of claim 1 , wherein the predicted outcome metric comprises an expected profit from replacing the item of the order with the replacement item.

6 . The method of claim 5 , wherein transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining the predicted outcome metric for the order satisfies one or more criteria comprises:
transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining the expected profit is less than a threshold amount.

7 . The method of claim 1 , wherein transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining the predicted outcome metric for the order satisfies one or more criteria comprises:
transmitting one or more instructions to the picker client device that display a notification message via a collection interface prompting the picker to refund the item rather than obtain the replacement item.

8 . The method of claim 1 , wherein transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining the predicted outcome metric for the order satisfies one or more criteria comprises:
transmitting one or more instructions to the picker client device that limit identification of the replacement item for the item to manual entry of an identifier of the replacement item by the picker.

9 . The method of claim 1 , wherein determining the predicted outcome metric for replacing the item with another item by applying the outcome prediction model to the item, order features for the order, and user features of the user comprises:
determining the predicted outcome metric for replacing the item with the replacement item by applying the outcome prediction model to the item, a replacement item, order features for the order, and user features of the user.

10 . The method of claim 9 , wherein transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining that the predicted outcome metric for the order satisfies one or more criteria comprises:
transmitting one or more instructions to the picker client device that prevent selection of the replacement item by the picker in response to determining the predicted outcome metric for the order satisfies one or more criteria.

11 . The method of claim 9 , wherein transmitting one or more instructions to the picker client device of the picker that modify one or more collection interfaces to increase the difficulty of identifying the replacement item for the item in response to determining that the predicted outcome metric for the order satisfies one or more criteria comprises:
transmitting one or more instructions to the picker client device that display a set of validated replacement items to the picker, the outcome prediction model determining a predicted outcome metric for the order including a validated replacement item when applied to the item, order features for the order, the validated replacement item, and user features of the user.

12 . A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps comprising:
receiving an order from a user, the order identifying one or more items and a retailer from which the one or more items are to be obtained; receiving an indication from a picker that an item included in the order is unavailable at the retailer; determining a predicted outcome metric for replacing the item wit

Assignees

• Maplebear Inc

Inventors

• Knight Benjamin
• Maheshwari Saumitra
• Braunstein Jennie
• Johnson Darren
• Sanchez Kenneth Jason
• Billman Christopher

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06Q10/087

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G06Q30/0631 Primary

  Recommending goods or services · CPC title