Automated and self-service item kiosk

Patent metadata

Field	Value
Publication number	US-11922386-B2
Application number	US-202117199883-A
Country	US
Kind code	B2
Filing date	Mar 12, 2021
Priority date	Mar 12, 2021
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

Disclosed are systems, methods, and apparatus of an automated and self-service kiosk that allows customers to select inventory items available from the kiosk and walk or move away with selected inventory item(s) without having to process payment, identify the inventory item(s), or provide any other form of checkout. After a customer has picked one or more items and departed the kiosk, the picked items are determined and the customer charged for the items. For example, one or more of detected weight changes measured at the kiosk and/or images generated at the kiosk may be used to identify items picked by the customer from the kiosk.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
at a kiosk:
detecting a unique identifier corresponding to an account;
in response to detecting the unique identifier, verifying that the unique identifier is valid and that access to the kiosk is to be provided;

in response to verifying that the unique identifier is valid and without first receiving an indication of an item to be retrieved from the kiosk or charging the account for a purchase or removal of an item from the kiosk, initiating a session during which access to a plurality of items contained within the kiosk is available;
recording, during the session, weight sensor data from a plurality of weight sensors within the kiosk;
recording, during the session, image data from a camera coupled to the kiosk;
determining that access to the plurality of items within the kiosk has ended; and
in response to determining that the access has ended:
completing the session;
determining, based at least in part on weight sensor data from a first weight sensor of the plurality of weight sensors, a first weight change as a first difference between a first weight measured by the first weight sensor during the session and a second weight measured by the first weight sensor during the session;
determining, based at least in part on the first weight change and an area within the kiosk associated with the first weight change, both a first action corresponding to the first weight change and a first item type of a first item involved in the first action;
updating an item pick list for the session based at least in part on the first action and the first item type;
updating an inventory count for the first item type at the kiosk; and
updating the account to indicated the first action and the first item type of the first item involved in the first action during the session at the kiosk.

2. The computer-implemented method of claim 1 , further comprising:
at the kiosk:
determining, based at least in part on weight sensor data from a second weight sensor of the plurality of weight sensors, a second weight change as a second difference between a third weight measured by the second weight sensor during the session and a fourth weight measured by the second weight sensor during the session;
determining that a second item type of a second item cannot be determined from the second weight change;
in response to determining that the second item type cannot be determined from the second weight change:
obtaining image data generated by the camera and corresponding to a time at which the second weight change occurred during the session; and
generating event data that includes at least the second weight change, the image data, and an area identifier indicating an area within the kiosk at which the second weight change occurred;

sending, to a remote computer resource, the event data;

at the remote computer resource:
processing the image data to determine one or more potential item types of the second item represented in the image data;
determining, based at least in part on the one or more potential item types, the second weight change, and the area identifier, the second item type of the second item corresponding to the second weight change; and
sending, to the kiosk, an indication of the second item type of the second item.

3. The computer-implemented method of claim 1 , further comprising:
at the kiosk:
determining, based at least in part on weight sensor data from a second weight sensor of the plurality of weight sensors, a second weight change as a second difference between a third weight measured by the second weight sensor during the session and a fourth weight measured by the second weight sensor during the session;
determining that a second item type of a second item cannot be determined from the second weight change;
in response to determining that the second item type cannot be determined from the second weight change:
obtaining image data generated by the camera and corresponding to a time at which the second weight change occurred during the session; and
generating event data that includes at least the second weight change, the image data, and an area identifier indicating an area within the kiosk at which the second weight change occurred;

sending, to a remote computer resource, the event data;

at the remote computer resource:
determining that the second item cannot be determined from the event data;
sending the event data for a manual review to determine the second item type of the second item corresponding to the second weight change;
receiving, from the manual review, an indication of the second item type; and
sending, to the kiosk, the indication of the second item type of the second item.

4. The computer-implemented method of claim 3 , further comprising:
providing at least a portion of the event data and the indication of the second item type as feedback to a machine learning model to improve an item recognition of the machine learning model.

5. The computer-implemented method of claim 1 , wherein:
the first action is a pick of an item of the first item type; and
updating the item pick list includes indicating the item of the first item type and a weight change determined for the first action.

6. The computer-implemented method of claim 1 , wherein determining both the first action and the first item type is further based at least in part on:
stored inventory data maintained in a data store of the kiosk, the stored inventory data indicating the first item type corresponding to the area and an average stored item weight of the first item type.

7. The computer-implemented method of claim 1 ,
wherein:
the first action is a pick of the first item of the first item type from a first area within the kiosk, wherein the first area corresponds to the first weight sensor of the plurality of weight sensors; and
updating the item pick list includes incrementing an item count for the first item type;

the method further comprising:
at the kiosk:
determining, based at least in part on weight sensor data from a second weight sensor, a second weight change as a second difference between a third weight measured by the second weight sensor during the session and a fourth weight measured by the second weight sensor during the session;
determining, based at least in part on the second weight change, a second action of a place of an item of the first item type to a second area within the kiosk, wherein the second area corresponds to the second weight sensor of the plurality of weight sensors;
updating the item pick list to decrement the item count for the first item type; and
indicating that the second area includes mixed item types.

8. The computer-implemented method of claim 1 , wherein determining both the first action and the first item type, further includes:
determining, from a data store at the kiosk, a stored item weight for a first item type associated with a first area corresponding to the first weight sensor;
determining that the first weight change is within a range of the stored item weight for the first item type; and
in response to determining that the first weight change is within the range of the stored item weight, determining that the first item of the first item type was involved in the first action.

9. The computer-implemented method of claim 1 , further comprising:
at the kiosk:
determining, based at least in part on weight sensor data from a second weight sensor of the plurality of weight sensors, a second weight change as a second difference between a third weight measured by the second weight sensor and a fourth weight measured by the second weight sensor;
determining, based at least in p

Assignees

• Amazon Tech Inc

Inventors

• Sasidharakurup Subhash
• Hariram Srinivasan
• Oberoi Mehakinder Singh
• Pal Ashish
• Jain Rajesh
• Sivadas Shanoop
• Singh Himanshu
• Dubhashi Aniket Nagesh
• Vaidya Vinay P
• Das Debasish

Classifications

• G06Q20/18 Primary

  involving self-service terminals [SST], vending machines, kiosks or multimedia terminals · CPC title

• G06Q10/087

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G07F11/62 Primary

  in which the articles are stored in compartments in fixed receptacles · CPC title

• G07F9/026 Primary

  for alarm, monitoring and auditing in vending machines or means for indication, e.g. when empty · CPC title

• H04W12/06

  Authentication · CPC title