Shopping basket monitoring using computer vision and machine learning

Patent metadata

Field	Value
Publication number	US-12033481-B2
Application number	US-201916561950-A
Country	US
Kind code	B2
Filing date	Sep 5, 2019
Priority date	Sep 7, 2018
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

A system for monitoring shopping baskets (e.g., baskets on human-propelled carts, motorized carts, or hand-carried baskets) can include a computer vision unit that can image a surveillance region (e.g., an exit to a store), determine whether a basket is empty or loaded with merchandise, and assess a potential for theft of the merchandise. The computer vision unit can include a camera and an image processor programmed to execute a computer vision algorithm to identify shopping baskets and determine a load status of the basket. The computer vision algorithm can comprise a neural network. The system can identify an at least partially loaded shopping basket that is exiting the store, without indicia of having paid for the merchandise, and execute an anti-theft action, e.g., actuating an alarm, notifying store personnel, activating a store surveillance system, activating an anti-theft device associated with the basket (e.g., a locking shopping cart wheel), etc.

First claim

What is claimed is:

1. An anti-theft system comprising:
a computer vision unit (CVU) configured to image a region of a facility, the region on a path between a pay point and an exit of the facility, the CVU comprising:
a camera;
a radio frequency (RF) communication node; and
an image processor; and

a human-propelled, wheeled cart comprising:
a basket configured to hold merchandise;
a wheel comprising a brake configured to inhibit movement of the cart when the brake is actuated; and
an RF cart transceiver configured to communicate with the RF communication node of the CVU and the brake,

wherein the anti-theft system, using the image processor, is programmed to analyze images of the region of the facility obtained by the camera to detect a pushout theft condition, wherein detecting the pushout theft condition comprises:
determining that the basket of the cart is at least partially loaded with merchandise; and
determining that the cart is approaching an exit of the facility;

wherein the anti-theft system is further programmed to generate, based at least partly on sensed motion of the cart in said region, data that distinguishes the cart from other carts in the facility;
wherein, at least partly in response to detecting the pushout theft condition, the anti-theft system, using the RF communication node, is configured to use the data that distinguishes the cart from other carts to communicate an anti-theft command to the RF cart transceiver of the cart.

2. The anti-theft system of claim 1 , wherein the CVU is further configured to:
communicate with a payment point of the facility; and
receive an indication from the payment point that a payment has not been made for the merchandise in the basket of the cart,
wherein the indication is received prior to communication of the anti-theft command to the RF cart transceiver.

3. The anti-theft system of claim 1 , wherein the image processor is programmed to apply a neural network to the images obtained by the camera.

4. The anti-theft system of claim 1 , wherein the image processor is further programmed to determine a path of the cart in the region of the facility.

5. The anti-theft system of claim 1 , wherein the CVU is further programmed to store the images of the region in a remote, non-transitory computer storage medium.

6. The anti-theft system of claim 1 , wherein the camera, the RF communication node, and the image processor are disposed in a housing configured to be mounted to a structure in the facility.

7. The anti-theft system of claim 1 , wherein the camera and the RF communication node are disposed in a housing configured to be mounted to a structure in the facility and the image processor is disposed remotely from the housing.

8. The anti-theft system of claim 1 , wherein the facility comprises a retail store and the human-propelled cart comprises a shopping cart.

9. The anti-theft system of claim 8 , wherein the region comprises the exit.

10. A method of reducing theft of merchandise from a retail store, the method comprising:
under control of an anti-theft system comprising computer hardware:
obtaining images of a region of the retail store, the region on a path between a pay point and an exit of the retail store;
identifying, from the images, presence of a shopping basket in the region;
determining, from the images, a load status indicative of whether the shopping basket is at least partially loaded with merchandise;
receiving payment information indicative of whether the merchandise in the shopping basket has been paid for;
generating, based at least partly on sensed motion of the shopping basket in the region, data that distinguishes the shopping basket from other shopping baskets in the retail store; and
communicating, based at least partly on the load status and using the data that distinguishes the shopping basket from other shopping baskets, an anti-theft command.

11. The method of claim 10 , wherein the identifying or the determining are performed using a neural network.

12. The method of claim 10 , further comprising determining, from the images, a path of the shopping basket in the region.

13. The method of claim 10 , wherein receiving payment information comprises:
obtaining second images of a payment point; and
determining, from the second images, whether the shopping basket passed the payment point, spent more than a threshold time near the payment point, interacted with a store attendant, or accessed a payment system at the payment point.

14. The method of claim 10 , further comprising determining a unicast address associated with a radio frequency (RF) receiver associated with the shopping basket based at least partly on said sensed motion.

15. The method of claim 10 , wherein communicating the anti-theft command comprises communicating to a transceiver associated with the shopping basket, to a checkout barrier, to a brake associated with a wheel associated with the shopping basket, or to a video surveillance system of the retail store.

16. The method of claim 10 , wherein the anti-theft command comprises a command to lock or brake a wheel associated with the shopping basket, a command to actuate an alarm or a warning, or a command to store personnel that a theft situation is occurring.

17. The method of claim 10 , wherein the shopping basket is associated with a wheeled, human-propelled shopping cart.

18. The method of claim 17 , wherein the shopping basket comprises a wheel having a brake, and the anti-theft command comprises a command to actuate the brake.

19. The method of claim 10 , wherein the shopping basket is associated with a handheld shopping basket.

20. The method of claim 10 , further comprising:
classifying the images of the region of the retail store to annotate shopping baskets or the load status of the shopping baskets so as to provide a set of training images; and
training a machine learning algorithm using the set of training images.

21. The anti-theft system of claim 1 , wherein the exit comprises an exit door.

22. The anti-theft system of claim 1 , wherein the sensed motion of the cart comprises a velocity of the cart.

23. The anti-theft system of claim 1 , wherein the sensed motion of the cart comprises a heading of the cart.

24. The anti-theft system of claim 1 , wherein the data that distinguishes the cart from other carts comprises a unicast address associated with the cart.

25. The anti-theft system of claim 24 , wherein the anti-theft system is programmed to determine the unicast address at least partly by detecting a correlation between cart motion determined from said images and cart motion detected by the cart.

26. The anti-theft system of claim 1 , wherein the data that distinguishes the cart from other carts is based on correlations between (1) cart motion detected in said region based on images captured by the CVU, and (2) cart motion detected by carts in said region.

27. The anti-theft system of claim 1 , wherein the anti-theft command comprises a command to actuate the brake of the wheel.

28. The method of claim 10 , wherein the sensed motion of the shopping basket comprises a velocity of the shopping basket.

29. The method of claim 10 , wherein the data that distinguishes the shopping basket from other shopping baskets comprises a unicast address associated with the shopping basket.

Assignees

• Gatekeeper Systems Inc

Inventors

• Ramanathan Narayanan V
• Carter Scott J
• Hannah Stephen E
• James Jesse M
• Johnson Jack L
• Harling Robert M

Classifications

• B62B3/1404

  Means for facilitating stowing or transporting of the trolleys; Antitheft arrangements (B62B5/0423 takes precedence) · CPC title

• B62B5/0404

  automatic · CPC title

• G08B13/19652

  Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm · CPC title

• G08B13/19613 Primary

  Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion · CPC title

• G08B13/19602 Primary

  Image analysis to detect motion of the intruder, e.g. by frame subtraction · CPC title