Systems and methods of identifying individual retail products in a product storage area based on an image of the product storage area

Patent metadata

Field	Value
Publication number	US-2025232603-A1
Application number	US-202519098571-A
Country	US
Kind code	A1
Filing date	Apr 2, 2025
Priority date	Oct 11, 2022
Publication date	Jul 17, 2025
Grant date	—

Abstract

Systems and methods of monitoring inventory of a product storage facility include an image capture device configured to move about the product storage areas of the product storage facility and capture images of the product storage areas from various angles. A computing device coupled to the image capture device obtains the images of the product storage areas captured by the image capture device and processes the obtained images of the product storage areas to detect individual products captured in the obtained images. Based on detection of the individual products captured in the images, the computing device analyzes each of the obtained images to extract meta data from the packaging the individual products to detect one more keywords and determine the locations of the detected keywords on the packaging, and then utilize this information to predict an identity of the products associated with the packaging.

First claim

1 .- 20 . (canceled)

21 . A system for detecting and recognizing products at product storage areas of a product storage facility, the system comprising:
an image capture device configured to capture at least one image of one or more products arranged in a product storage area of the product storage facility; a computing device communicatively coupled to the image capture device, the computing device including a control circuit configured to: detect an individual product in the at least one image of the one or more products; extract, from the at least one image of the one or more products, metadata associated with the individual product; identify a keyword displayed on the individual product based on the extracted metadata; and identify the individual product based at least partially on a location of the keyword displayed on the individual product.

22 . The system of claim 21 , wherein the image capture device includes a camera integrated within a motorized robotic unit configured to move about the product storage facility.

23 . The system of claim 21 , wherein the extracted metadata includes one or more characters or text extracted by performing optical character recognition on the at least one image of the one or more products.

24 . The system of claim 23 , wherein the control circuit is further configured to identify the keyword displayed on the individual product based on a height or width of the one or more characters or text in the extracted metadata.

25 . The system of claim 21 , wherein the control circuit is further configured to generate, within the at least one image of the one or more products, a virtual bounding box surrounding the keyword displayed on the individual product.

26 . The system of claim 25 , wherein control circuit is further configured to determine the location of the keyword displayed on the individual product based on center coordinates of the virtual bounding box.

27 . The system of claim 25 , wherein control circuit is further configured to detect a height of the keyword displayed on the individual product based on at least one dimension of the virtual bounding box, wherein the individual product is identified based on a combination of the height of the keyword displayed on the individual product and the location of the keyword displayed on the individual product.

28 . The system of claim 25 , wherein control circuit is further configured to detect a width of the keyword displayed on the individual product based on at least one dimension of the virtual bounding box, and wherein the individual product is identified based on a combination of the width of the keyword displayed on the individual product and the location of the keyword displayed on the individual product.

29 . The system of claim 21 , wherein control circuit is further configured to detect an exterior color of the individual product, and wherein the individual product is identified based on a combination of the exterior color of the individual product and the location of the keyword displayed on the individual product.

30 . The system of claim 21 , wherein control circuit is further configured to detect a size of the individual product, and wherein the individual product is identified based on a combination of the size of the individual product and the location of the keyword displayed on the individual product.

31 . The system of claim 21 , wherein control circuit is further configured to detect a shape of the individual product, and wherein the individual product is identified based on a combination of the shape of the individual product and the location of the keyword displayed on the individual product.

32 . The system of claim 21 , wherein control circuit is further configured to detect a separate keyword displayed on the individual product, and wherein the individual product is identified based on relative locations of the separate keyword and the keyword displayed on the individual product.

33 . A method comprising:
capturing at least one image of one or more products arranged in a product storage area of a product storage facility; detecting an individual product in the at least one image of the one or more products; extracting, from the at least one image of the one or more products, metadata associated with the individual product; identifying a keyword displayed on the individual product based on the extracted metadata; and identifying the individual product based at least partially on a location of the keyword displayed on the individual product.

34 . The method of claim 33 , further comprising:
generating, within the at least one image of the one or more products, a virtual bounding box surrounding the keyword displayed on the individual product.

35 . The method of claim 34 , wherein the location of the keyword displayed on the individual product is determined based on center coordinates of the virtual bounding box.

36 . The method of claim 34 , further comprising detecting a height of the keyword displayed on the individual product based on at least one dimension of the virtual bounding box, wherein the individual product is identified based on a combination of the height of the keyword displayed on the individual product and the location of the keyword displayed on the individual product.

37 . The method of claim 34 , further comprising detecting a width of the keyword displayed on the individual product based on at least one dimension of the virtual bounding box, wherein the individual product is identified based on a combination of the width of the keyword displayed on the individual product and the location of the keyword displayed on the individual product.

38 . The method of claim 33 , further comprising detecting an exterior color of the individual product, and wherein the individual product is identified based on a combination of the exterior color of the individual product and the location of the keyword displayed on the individual product.

39 . The method of claim 33 , further comprising detecting a size or shape of the individual product, and wherein the individual product is identified based on a combination of the size or shape of the individual product and the location of the keyword displayed on the individual product.

40 . A method comprising:
capturing at least one image of one or more products arranged in a product storage area of a product storage facility; detecting an individual product in the at least one image of the one or more products; extracting, from the at least one image of the one or more products, metadata associated with the individual product; identifying a keyword displayed on the individual product based on the extracted metadata; and identifying the individual product based at least partially on a height or width of the keyword displayed on the individual product.

Assignees

• Walmart Apollo Llc

Inventors

• Zhang Lingfeng
• Zhang Han
• Pachauri Abhinav
• Jhunjhunwala Amit
• Ghosh Ashlin
• Jade Avinash M
• Balusu Raghava
• Muktevi Srinivas
• Yuan Mingquan
• Duan Zhaoliang
• Huang Zhiwei
• Mao Tianyi

Classifications

• G06V20/36

  Indoor scenes · CPC title

• G06Q10/08 Primary

  Logistics, e.g. warehousing, loading or distribution; Inventory or stock management · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06Q10/087

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G06V30/153 Primary

  using recognition of characters or words · CPC title