Machine learning assisted alerts for item picking

Patent metadata

Field	Value
Publication number	US-12412151-B2
Application number	US-202318498016-A
Country	US
Kind code	B2
Filing date	Oct 30, 2023
Priority date	Oct 30, 2023
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

A machine-learned predictive model is trained to predict potential for customer complaint. The model is part of an online concierge system. The online concierge system accesses a customer order that includes one or more items. The online concierge system determines input data for an item of the one or more items. The online concierge system determines a prediction value associated with potential for customer complaint for the item by applying the machine-learned prediction model to the input data. The online concierge system provides the prediction value to a picker client device associated with a picker who is assigned the item. The picker client device presents an alert to the picker based in part on the prediction value, and the alert includes a message that is customized to mitigate a cause of potential customer complaint for the item.

First claim

What is claimed is:

1. A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor of a computer system, cause the processor to perform steps comprising:
accessing a customer order that includes one or more items;
determining input data for an item of the one or more items;
determining, by the computer system, a prediction value associated with potential for a customer complaint for the item by the computer system applying a machine-learned prediction model to the input data, wherein the machine-learned prediction model was trained by:
accessing a training data set including item identifiers for a plurality of items including the item and instances of customer complaint for the plurality of items,
applying the machine-learned prediction model to the training data to generate a training output,
backpropagating one or more error terms obtained from one or more loss functions to update a set of parameters of the machine-learned prediction model based, and one or more of the error terms are based on a difference between a label applied to an item of the training data and a predicted probability of receiving a customer complaint for that item, and
stopping the backpropagation after the one or more loss functions satisfy one or more criteria; and

providing the prediction value to a picker client device associated with the item, wherein providing the prediction value to the picker client device causes the picker client device to present an alert, and wherein the alert includes a message that is customized to mitigate a cause of potential customer complaint for the item.

2. The computer program product of claim 1 , wherein determining input data associated with the item comprises determining an item identifier that identifies characteristics of the item, a customer identifier that identifies characteristics of a customer associated with a client device that provided the customer order, and a picker identifier that identifies characteristics of the picker.

3. The computer program product of claim 2 , wherein the training data set used to train the machine-learned prediction model includes item identifiers that identify respective characteristics of the plurality of items, customer identifiers that identify respective characteristics of customers, and picker identifiers that identify characteristics of pickers.

4. The computer program product of claim 1 , the non-transitory computer readable storage medium having additional instructions encoded thereon that, when executed by the processor, cause the processor to perform steps comprising:
selecting a message from a plurality of message types based in part on a type of issue associated with the item; and
providing the selected message to the picker client device.

5. The computer program product of claim 4 , wherein the type of issue is one or more of: item missing, wrong item, poor replacement, rotten, expired, damaged, temperature spoiled, or uncategorized.

6. The computer program product of claim 1 , the non-transitory computer readable storage medium having additional instructions encoded thereon that, when executed by the processor, cause the processor to perform steps comprising:
determining a type of issue associated with the item; and
selecting the machine-learned prediction model associated with the determined type of issue from a plurality of machine-learned prediction models that are associated with different types of issues.

7. The computer program product of claim 1 , wherein the machine-learned prediction model is further trained using one or more guardrail metrics associated with the plurality of items, wherein a guardrail metric includes mean picking time per order.

8. The computer program product of claim 1 , wherein providing the prediction value to the picker client device is responsive to the prediction value exceeding a threshold value.

9. The computer program product of claim 8 , wherein providing the prediction value to the picker client device further comprises, responsive to the prediction value exceeding the threshold value, the picker client device selects the message from a plurality of messages.

10. A method, performed at a computer system comprising a processor and a non-transitory computer readable medium, comprising:
accessing a customer order that includes one or more items;
determining input data for an item of the one or more items;
determining, by the computer system, a prediction value associated with potential for customer complaint for the item by the computer system applying a machine-learned prediction model to the input data, wherein the machine-learned prediction model was trained by:
accessing a training data set including item identifiers for a plurality of items including the item and instances of customer complaint for the plurality of items,
backpropagating one or more error terms obtained from one or more loss functions to update a set of parameters of the machine-learned prediction model based, and one or more of the error terms are based on a difference between a label applied to an item of the training data and a predicted probability of receiving a customer complaint for that item, and
stopping the backpropagation after the one or more loss functions satisfy one or more criteria; and

providing the prediction value to a picker client device associated with the item, wherein providing the prediction value to the picker client device causes the picker client device to present an alert, and wherein the alert includes a message that is customized to mitigate a cause of potential customer complaint for the item.

11. The method of claim 10 , wherein determining input data associated with the item comprises determining an item identifier that identifies characteristics of the item, a customer identifier that identifies characteristics of a customer associated with a client device that provided the customer order, and a picker identifier that identifies characteristics of the picker.

12. The method of claim 11 , wherein the training data set used to train the machine-learned prediction model include item identifiers that identify respective characteristics of the plurality of items, customer identifiers that identify respective characteristics of customers, and picker identifiers that identify characteristics of pickers.

13. The method of claim 10 , further comprising:
selecting a message from a plurality of message types based in part on a type of issue associated with the item; and
providing the selected message to the picker client device.

14. The method of claim 13 , wherein the type of issue is one or more of: item missing, wrong item, poor replacement, rotten, expired, damaged, temperature spoiled, or uncategorized.

15. The method of claim 10 , further comprising:
determining a type of issue associated with the item; and
selecting the machine-learned prediction model associated with the determined type of issue from a plurality of machine-learned prediction models that are associated with different types of issues.

16. The method of claim 10 , wherein the machine-learned prediction model is further trained using one or more guardrail metrics associated with the plurality of items, wherein a guardrail metric includes mean picking time per order.

17. The method of claim 10 , wherein providing the prediction value to the picker client device comprises, responsive to the prediction value exceeding a threshold value, causing the picker client device is configured to present

Assignees

• Maplebear Inc

Inventors

• Li Shang
• Sinha Ashish
• Selvam Krishna Kumar
• Xi Qi
• Darvishzadeh Amirali
• Zandman David
• Billman Christopher

Classifications

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title