Systems and methods for reducing false identifications of products

Patent metadata

Field	Value
Publication number	US-2024273463-A1
Application number	US-202318168174-A
Country	US
Kind code	A1
Filing date	Feb 13, 2023
Priority date	Feb 13, 2023
Publication date	Aug 15, 2024
Grant date	—

Abstract

In some embodiments, apparatuses and methods are provided herein useful to processing captured images. In some embodiments, there is provided a system for processing captured images of objects at a product storage facility including a trained machine learning model; and a control circuit. The control circuit may identify a product identifier associated with an object in a captured image; generate predicted product identifiers associated with the object in the captured image based on text identified from the object in the captured image; aggregate the predicted product identifiers; determine a feature of the objects associated with the aggregated predicted product identifiers; determine one or more confusing product identifiers based on a determination of the aggregated predicted product identifiers being associated with the feature; and update a dataset with at least one of the one or more confusing product identifiers and images associated with the one or more confusing product identifiers.

First claim

What is claimed is:

1 . A system for processing captured images of objects at a product storage facility, the system comprising:
a trained machine learning model stored in a memory; and a control circuit executing the trained machine learning model to determine confusing product identifiers, wherein the confusing product identifiers correspond to objects that are at least one of textually similar and visually similar such that the objects can be potentially mis-identified with an incorrect product identifier, the control circuit configured to:
receive a plurality of captured images, wherein each captured image depicts at least one object for purchase at the product storage facility;
identify, for each captured image, a product identifier associated with an object in a captured image;
generate, for each captured image, predicted product identifiers associated with the object in the captured image based on text identified from the object in the captured image;
aggregate the predicted product identifiers associated with identical product identifiers;
determine a feature of the objects associated with the aggregated predicted product identifiers that is greater than a feature threshold;
determine one or more confusing product identifiers based on a determination of the aggregated predicted product identifiers being associated with the feature; and
update a dataset with at least one of the one or more confusing product identifiers and images associated with the one or more confusing product identifiers.

2 . The system of claim 1 , wherein the control circuit executing the trained machine learning model to determine the confusing product identifiers is further configured to identify, based on the updated dataset, correct product identifiers to associate with at least one of textually similar and visually similar objects depicted in the captured images.

3 . The system of claim 1 , further comprising:
one or more image capture devices configured to capture a plurality of images of objects at the product storage facility; and a database configured to store the plurality of images.

4 . The system of claim 3 , wherein at least one of the one or more image capture devices is coupled to a motorized robotic unit.

5 . The system of claim 3 , wherein the plurality of images comprise images that have not gone through object detection or object classification.

6 . The system of claim 1 , wherein the trained machine learning model comprises one or more machine learning models each trained to perform a corresponding operation executed by the control circuit to determine the confusing product identifiers.

7 . The system of claim 6 , wherein a first trained machine learning model of the one or more machine learning models is trained to perform the generation of the predicted product identifiers based on a determination of score values associated with stored product identifiers based on at least one or more steps comprising:
determine text associated with stored product identifiers that matches the most relative to other text associated with the stored product identifiers with the text identified from the object in the captured image; compare whether a location associated with the text identified from the object in the captured image matches with one or more locations associated with the most matching text associated with the stored product identifiers within a threshold range; and determine whether one or more of the stored product identifiers and the object in the captured image are associated with a matching presence of a first text and a matching absence of a second text, wherein the predicted product identifiers comprise those stored product identifiers having corresponding score values that are greater than a score threshold.

8 . The system of claim 7 , further comprising a database configured to store the stored product identifiers.

9 . The system of claim 6 , wherein a first trained machine learning model of the one or more machine learning models is trained to perform the determination of the feature of the objects associated with the aggregated predicted product identifiers based on metric learning algorithm.

10 . The system of claim 1 , wherein each captured image comprises metadata information determined by the control circuit.

11 . A method for processing captured images of objects at a product storage facility, the method comprising:
receiving, by a control circuit executing a trained machine learning model stored in a memory to determine confusing product identifiers, a plurality of captured images, wherein each captured image depicts at least one object for purchase at the product storage facility, and wherein the confusing product identifiers correspond to objects that are at least one of textually similar and visually similar such that the objects can be potentially mis-identified with an incorrect product identifier; identifying, by the control circuit executing the trained machine learning model and for each captured image, a product identifier associated with an object in a captured image; generating, by the control circuit executing the trained machine learning model and for each captured image, predicted product identifiers associated with the object in the captured image based on text identified from the object in the captured image; aggregating, by the control circuit executing the trained machine learning model, the predicted product identifiers associated with identical product identifiers; determining, by the control circuit executing the trained machine learning model, a feature of the objects associated with the aggregated predicted product identifiers that is greater than a feature threshold; determining, by the control circuit executing the trained machine learning model, one or more confusing product identifiers based on a determination of the aggregated predicted product identifiers being associated with the feature; and updating, by the control circuit executing the trained machine learning model, a dataset with at least one of the one or more confusing product identifiers and images associated with the one or more confusing product identifiers.

12 . The method of claim 11 , further comprising identifying, by the control circuit executing the trained machine learning model and based on the updated dataset, correct product identifiers to associate with at least one of textually similar and visually similar objects depicted in the captured images.

13 . The method of claim 11 , further comprising:
capturing, by one or more image capture devices, a plurality of images of objects at the product storage facility; and storing, by a database, the plurality of images.

14 . The method of claim 13 , wherein at least one of the one or more image capture devices is coupled to a motorized robotic unit.

15 . The method of claim 13 , wherein the plurality of images comprise images that have not gone through object detection or object classification.

16 . The method of claim 11 , wherein the trained machine learning model comprises one or more machine learning models each trained to perform a corresponding operation executed by the control circuit to determine the confusing product identifiers.

17 . The method of claim 16 , wherein the generating of the predicted product identifiers is based on determining score values associated with stored product identifiers based on at least one or more steps comprising:
determining, by a first trained machine learning model of the one or more machine learning models, text associated wi

Assignees

• Walmart Apollo Llc

Inventors

• Pachauri Abhinav
• Balusu Raghava
• Jade Avinash M
• Zhang Lingfeng
• Muktevi Srinivas
• Jhunjhunwala Amit
• Duan Zhaoliang

Classifications

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V30/1444 Primary

  Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields · CPC title

• G06V2201/07

  Target detection · CPC title