Systems and methods for assessing quality of retail products

Patent metadata

Field	Value
Publication number	US-12175476-B2
Application number	US-202217588922-A
Country	US
Kind code	B2
Filing date	Jan 31, 2022
Priority date	Jan 31, 2022
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

Systems and methods for assessing quality of and detecting defects in retail products include a housing having an interior, a product holding surface, an image capture device configured to capture an image of the product, a lighting element configured to provide lighting within the housing, an electronic database configured to store electronic data including reference image data associated with the product and representing the product in various sizes and various quality states, and a processor-based control circuit in communication with the first image capture device and the electronic database. The control circuit is configured to obtain image data representing the image of the product, obtain the reference image data, analyze the image data and the reference image data to identify the product and identify a size of a defect present on the product, and output a notification indicating whether or not the product is of acceptable quality.

First claim

What is claimed is:

1. A system for assessing quality of a plurality of products, the system comprising:
at least one image capture device configured to capture at least one image of at least one product of the plurality of products from at least one perspective, the products being perishable, consumable items selected from a group consisting of: food items, grocery items, beverage items, medication items, and dietary supplement items;
an electronic database configured to store electronic data including reference image data associated with the at least one product and representing the at least one product in various sizes and various quality states;
a processor-based control circuit in communication with at least one of the first image capture device and the electronic database, and configured to:
obtain image data representing the at least one image of the at least one product;
obtain, from the electronic database, the reference image data;
analyze the image data and the reference image data to:
identify the at least one product;
identify a type of a defect present on a surface of the at least one product;
identify a surface area of the defect present on the surface of the at least one product;
correlate the type of the defect and the surface area of the defect present on the surface of the at least one product to the type of the at least one product and an overall surface area of the at least one product to determine a defect severity level of the at least one product; and
correlate the determined defect severity level of the at least one product to a threshold defect severity level predetermined for the type of the at least one product to determine whether the at least one product is of acceptable quality for consumption by a consumer, and:
when the determined defect severity level of the at least one product is below the threshold defect severity level predetermined for the at least one product, output a notification for the at least one product indicating that the at least one product is of acceptable quality for consumption by the consumer; and
when the determined defect severity level of the at least one product is above the threshold defect severity level predetermined for the at least one product, output a notification for the at least one product indicating that the at least one product is not of acceptable quality for consumption by the consumer.

2. The system of claim 1 , wherein the control circuit is further programmed to analyze the image data and the reference image data to detect exterior contours of the at least one product to identify the size of the at least one product, the size of the at least one product including at least one of length, width, and height of the at least one product.

3. The system of claim 2 , wherein the control circuit is further programmed to:
determine a scale factor and a number of pixels representing the contours of the at least one product using a color card; and
translate the number of pixels representing the contours of the at least one product to physical dimensions of the at least one product.

4. The system of claim 1 , wherein the control circuit is further programmed to analyze the image data associated with the at least one product using a color card to white balance and color correct colors in the image data.

5. The system of claim 1 , wherein the control circuit is further programmed to:
analyze the image data and the reference image data to identify a type of defect present on the at least one product; and
output an indication of the type of defect on the at least one product identified by the control circuit.

6. The system of claim 1 , wherein the control circuit is further programmed to analyze the image data and the reference image data to determine an area of the defect present on the at least one product via a machine learning model, wherein the machine learning model is trained using the reference image data stored in the electronic database.

7. The system of claim 1 , further comprising:
a housing including an interior and an opening configured to allow placement of the at least one product into the interior of the housing;
a product holding surface located within the housing, wherein the product holding surface allows images to be taken through the product holding surface, wherein the product holding surface is configured to support the at least one product;
wherein the product holding surface includes one or more transparent stoppers sized and shaped to restrict the at least one product from moving on the product holding surface and hold the at least one product in a position such that the defect on the at least one product faces the at least one image capture device.

8. The system of claim 7 , further comprising at least one lighting element located within the housing and configured to provide lighting within the housing, wherein:
the housing includes at least one side wall facing the interior;
the at least side wall is illuminated by the at least one lighting element to provide a monochromatic background; and
the at least one lighting element, in response to a signal transmitted by the control circuit, changes a color of the at least one side wall.

9. The system of claim 7 , further comprising at least one lighting element located within the housing, wherein the at least one lighting element includes one or more status lights indicating an operational status of at least one of the control circuit, the at least one image capture device, and the at least one lighting element.

10. The system of claim 1 , wherein the type of defect present on the surface of the at least one product comprises at least one of: scars, shriveled ends, sunken areas, bruising, mold, and decay.

11. A method of assessing quality of a plurality of products, the method comprising:
capturing at least one image of at least one product of the plurality of products from at least one perspective by at least one image capture device, the products being perishable, consumable items selected from a group consisting of: food items, grocery items, beverage items, medication items, and dietary supplement items;
storing, in an electronic database, electronic data including reference image data associated with the at least one product and representing the at least one product in various sizes and various quality states;
by a processor-based control circuit that is in communication with at least one of the first image capture device and the electronic database:
obtaining image data representing the at least one image of the at least one product;
obtaining, from the electronic database, the reference image data;
analyzing the image data and the reference image data to:
identify the at least one product;
identify a type of a defect present on a surface of the at least one product;
identify a surface area of the defect present on the surface of the at least one product;

correlating the type of the defect and the surface area of the defect present on the surface of the at least one product to the type of the at least one product and an overall surface area of the at least one product to determine a defect severity level of the at least one product; and
correlating determined defect severity level of the at least one product to a threshold defect severity level predetermined for the type of the at least one product to determine whether the at least one product is of acceptable quality for consumption by a consumer; and:
when the determined defect severity level of the at least one product is below the threshold defect severity level predetermined for the at least one product, outputting a notification for the at least one product indicati

Assignees

• Walmart Apollo Llc

Inventors

• Sambasivan Lokesh Kumar
• Tendulkar Sandeep Jaywant
• Naik Shivani S
• Pushp Pushkar
• Chakraborty Soumabrata Arup
• Kushwaha Abhishek
• Caballero Perales Maxine
• Tilmon Chuck E
• Valenzuela Geronimo
• Baber Ray L
• Forsgren Daniel J
• Rorie Chuck S
• Sathyamurthy Raghuram
• Van Voorst Nicholas G

Classifications

• G06T7/001

  using an image reference approach · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T2207/30128

  Food products · CPC title

• G06T2207/10152

  Varying illumination · CPC title

• G06T2207/10024

  Color image · CPC title