Systems and methods for detecting text in images

Patent metadata

Field	Value
Publication number	US-11954928-B2
Application number	US-202318118658-A
Country	US
Kind code	B2
Filing date	Mar 7, 2023
Priority date	Nov 14, 2019
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

In some embodiments, apparatuses and methods are provided herein useful to detecting text in images. In some embodiments, a system for detecting text in images comprises a database configured to store images and a control circuit configured to retrieve an image, generate, based on the image, a collection of augmented images, detect characters in each of the augmented images, generate bounding boxes for the characters in each of augmented images, recognize the characters in each of the augmented images, select, based on the recognition of the characters in each of the augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the augmented images, detect, for the image, a color associated with the characters, and store, in the database, the image, the candidate characters, and the color associated with the characters.

First claim

What is claimed is:

1. A system for detecting text in images, the system comprising:
a database, wherein the database is configured to store images, candidate characters in the images, and a color associated with the candidate characters;
an image capture device, wherein the image capture device is configured to capture images of products, and
a control circuit coupled to the image capture device and the database, the control circuit configured to:
receive an image of a product captured by the image capture device;
detect characters on the product in the image;
detect a color associated with the characters on the product in the image;
recognize the characters on the product in the image;
recognize the color associated with the characters on the product in the image;
determine, based on accessing the database, a logo including the candidate characters on one of the products in one of the stored images;
determine, based on accessing the database, a color associated with the logo; and
verify the recognition of the characters on the product in the image based on a matching of the color associated with the characters on the product in the image and the color associated with the logo.

2. The system of claim 1 , wherein the control circuit is further configured to:
retrieve an image stored in the database;
generate, based on the image retrieved from the database, a collection of augmented images;
detect characters in each one of images in the collection of augmented images;
recognize the characters in each one of the images in the collection of augmented images; and
select, based on recognition of the characters in each one of the images in the collection of augmented images, the candidate characters.

3. The system of claim 2 , wherein the control circuit is further configured to select the candidate characters based on consistency of recognition of the characters in each one of the images in the collection of augmented images.

4. The system of claim 1 , wherein the logo includes one or more of a brand, an insignia, product packaging, and a symbol.

5. The system of claim 2 , wherein the control circuit is further configured to:
determine, based on the accessing of the database, a list of candidate logos, wherein the candidate logos match the color associated with the characters on the product in the image; and
select, from the list of candidate logos and based on the recognition of the characters on the product in the image, a type of product for the product in the image.

6. The system of claim 1 , wherein the logo includes one or more of a brand, an insignia, product packaging, and a symbol.

7. The system of claim 1 , wherein the database is further configured to store a data set based on the characters recognized by the control circuit.

8. The system of claim 1 , wherein the color associated with the characters is one or more of a color of the characters, a background color for the characters, color histogram within a bounding box, and a color within the bounding box.

9. The system of claim 2 , wherein the control circuit is configured to recognize the characters in each one of the images of the collection of augmented images based on one or more of optical character recognition (OCR) and an item detection model.

10. The system of claim 2 , wherein the collection of augmented images includes at least 50 images.

11. A method for detecting text in images, the method comprising:
storing, in a database, images, candidate characters in the images, and a color associated with the candidate characters;
capturing images of products by an image capture device;
by a control circuit coupled to the image capture device and the database:
receiving an image of a product captured by the image capture device;
detecting characters on the product in the image;
detecting a color associated with the characters on the product in the image;
recognizing the characters on the product in the image;
recognizing the color associated with the characters on the product in the image;
determining, based on accessing the database, a logo including the candidate characters on one of the products in one of the stored images;
determining, based on accessing the database, a color associated with the logo; and
verifying the recognition of the characters on the product in the image based on a matching of the color associated with the characters on the product in the image and the color associated with the logo.

12. The method of claim 11 , further comprising, by the control circuit:
retrieving an image stored in the database;
generating, based on the image retrieved from the database, a collection of augmented images;
detecting characters in each one of images in the collection of augmented images;
recognizing the characters in each one of the images in the collection of augmented images; and
selecting, based on recognition of the characters in each one of the images in the collection of augmented images, the candidate characters.

13. The method of claim 12 , further comprising, by the control circuit, selecting the candidate characters based on consistency of recognition of the characters in each one of the images in the collection of augmented images.

14. The method of claim 11 , wherein the logo includes one or more of a brand, an insignia, product packaging, and a symbol.

15. The method of claim 12 , further comprising, by the control circuit:
determining, based on the accessing of the database, a list of candidate logos, wherein the candidate logos match the color associated with the characters on the product in the image; and
selecting, from the list of candidate logos and based on the recognition of the characters on the product in the image, a type of product for the product in the image.

16. The method of claim 11 , wherein the logo includes one or more of a brand, an insignia, product packaging, and a symbol.

17. The method of claim 11 , further comprising storing in the database a data set based on the characters recognized by the control circuit.

18. The method of claim 11 , wherein the color associated with the characters is one or more of a color of the characters, a background color for the characters, color histogram within a bounding box, and a color within the bounding box.

19. The method of claim 12 , further comprising, by the control circuit, recognizing the characters in each one of the images of the collection of augmented images based on one or more of optical character recognition (OCR) and an item detection model.

20. The method of claim 12 , wherein the collection of augmented images includes at least 50 images.

Assignees

• Walmart Apollo Llc

Inventors

• Liu Haining
• Zhu Feiyun
• Hammer Jon
• Reagan Ryan B
• Yu Pingjian
• Xiao Zhichun
• Zhang Yuqi
• Liu Yao

Classifications

• G06V30/18105 Primary

  related to colour · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/56

  relating to colour · CPC title

• G06V20/62

  Text, e.g. of license plates, overlay texts or captions on TV images · CPC title

• G06V30/10

  Character recognition · CPC title