Systems and methods for detecting text in images

Patent metadata

Field	Value
Publication number	US-11620839-B2
Application number	US-202017097734-A
Country	US
Kind code	B2
Filing date	Nov 13, 2020
Priority date	Nov 14, 2019
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

In some embodiments, apparatuses and methods are provided herein useful to detecting text in images. In some embodiments, a system for detecting text in images comprises a database configured to store images and a control circuit configured to retrieve an image, generate, based on the image, a collection of augmented images, detect characters in each of the augmented images, generate bounding boxes for the characters in each of augmented images, recognize the characters in each of the augmented images, select, based on the recognition of the characters in each of the augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the augmented images, detect, for the image, a color associated with the characters, and store, in the database, the image, the candidate characters, and the color associated with the characters.

First claim

What is claimed is:

1. A system for detecting text in images, the system comprising:
a database, wherein the database is configured to store images; and
a control circuit, wherein the control circuit is communicatively coupled to the database, and wherein the control circuit is configured to:
retrieve, from the database, an image;
generate, based on the image, a collection of augmented images;
detect characters in each of the images of the collection of augmented images;
generate bounding boxes for the characters in each of the images of the collection of augmented images;
recognize the characters in each of the images of the collection of augmented images;
select, based on the recognition of the characters in each of the images of the collection of augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the images of the collection of augmented images;
detect, for the image, a color associated with the characters;
store, in the database, the image, the candidate characters, and the color associated with the characters; and
an image capture device, wherein the image capture device is configured to capture images of carts, and wherein the control circuit is further configured to:
receive one of the images of the carts;
detect characters on a product in the one of the images;
detect a color associated with the characters on the product in the one of the images;
recognize the characters on the product in the one of the images;
determine, based on accessing the database, a logo including the characters on the product in the one of the images;
determine, based on accessing the database, a color associated with the logo; and
verify the recognition of the characters on the product in the one of the images based on a matching of the color associated with the characters on the product in the one of the images and the color associated with the logo.

2. The system of claim 1 , wherein the logo includes one or more of a brand, an insignia, product packaging, and a symbol.

3. The system of claim 1 , wherein the database is further configured to store a data set based on the recognized characters.

4. The system of claim 1 , wherein the color associated with the characters is one or more of a color of the characters, a background color for the characters, color histogram within a bounding box, and a color within the bounding box.

5. The system of claim 1 , wherein the control circuit recognizes the characters in each of the images of the collection of augmented images based on one or more of optical character recognition (OCR) and an item detection model.

6. The system of claim 1 , wherein the collection of augmented images includes at least 50 images.

7. A system for detecting text in images, the system comprising:
a database, wherein the database is configured to store images;
a control circuit, wherein the control circuit is communicatively coupled to the database, and wherein the control circuit is configured to:
retrieve, from the database, an image;
generate, based on the image, a collection of augmented images;
detect characters in each of the images of the collection of augmented images;
generate bounding boxes for the characters in each of the images of the collection of augmented images;
recognize the characters in each of the images of the collection of augmented images;
select, based on the recognition of the characters in each of the images of the collection of augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the images of the collection of augmented images;
detect, for the image, a color associated with the characters;
store, in the database, the image, the candidate characters, and the color associated with the characters; and
an image capture device, wherein the image capture device is configured to capture images of carts, and wherein the control circuit is further configured to:
receive one of the images of the carts;
detect characters on a product in the one of the images;
detect a color associated with the characters on the product in the one of the images;
determine, based on accessing the database, a list of candidate logos, wherein the candidate logos match the color associated with the characters on the product in the one of the images;
recognize at least one of the characters on the product in the one of the images; and
select, from the list of candidate logos based on the recognition of the at least one of the characters, a type of product for the product in the one of the images.

8. The system of claim 7 , wherein the logo includes one or more of a brand, an insignia, product packaging, and a symbol.

9. A method for detecting text in images, the method comprising:
storing, in a database, images;
retrieving, by a control circuit from the database, an image;
generating, by the control circuit based on the image, a collection of augmented images;
detecting, by the control circuit, characters in each of the images of the collection of augmented images;
generating, by the control circuit, bounding boxes for the characters in each of the images of the collection of augmented images;
recognizing, by the control circuit, the characters in each of the images of the collection of augmented images;
selecting, by that control circuit based on the recognition of the characters in each of the images of the collection of augmented images, candidate characters, wherein the candidate characters are selected based on consistency of the recognition of the characters in each of the images of the collection of augmented images;
detecting, by the control circuit for the image, a color associated with the characters; and
storing, in the database, the image, the candidate characters, and the color associated with the characters;
capturing, by an image capture device, images of carts;
receiving, by the control circuit, one of the images of the carts;
detecting, by the control circuit, characters on a product in the one of the images;
recognizing, by the control circuit, the character on the product in the one of the images;
determining, by the control circuit based on accessing the database, a logo including the characters on the product in the one of the images;
determining, by the control circuit based on accessing the database, a color associated with the logo; and
verifying, by the control circuit, the recognition of the characters on the product in the one of the images based on a matching of the color associated with the characters on the product in the one of the images and the color associated with the logo.

10. The method of claim 9 , wherein the logo includes one or more of a brand, an insignia, product packaging, and a symbol.

11. The method of claim 9 , wherein the database is further configured to store a data set based on the recognized characters.

12. The method of claim 9 , wherein the color associated with the characters is one or more of a color of the characters, a background color for the characters, a color histogram within a bounding box, and a color within the bounding box.

13. The method of claim 9 , wherein the control circuit recognizes the characters in each of the images of the collection of augmented images based on one or more of optical character recognition (OCR) and an item detection model.

14. The method of claim 9 , wherein the collection of augmented images includes at least 50 images.

15. A method for detecting text in

Assignees

• Walmart Apollo Llc

Inventors

• Liu Haining
• Zhu Feiyun
• Hammer Jon
• Reagan Ryan B
• Yu Pingjian
• Xiao Zhichun
• Zhang Yuqi
• Liu Yao

Classifications

• G06V20/62 Primary

  Text, e.g. of license plates, overlay texts or captions on TV images · CPC title

• G06V30/18105 Primary

  related to colour · CPC title

• G06V10/56

  relating to colour · CPC title

• G06V30/10

  Character recognition · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title