Systems and methods for image preprocessing to improve accuracy of object recognition

Patent metadata

Field	Value
Publication number	US-10157331-B1
Application number	US-201815915583-A
Country	US
Kind code	B1
Filing date	Mar 8, 2018
Priority date	Mar 8, 2018
Publication date	Dec 18, 2018
Grant date	Dec 18, 2018

Abstract

The present disclosure relates to image preprocessing to improve object recognition. In one implementation, a system for preprocessing an image for object recognition may include at least one memory storing instructions and at least one processor configured to execute the instructions to perform operations. The operations may include receiving the image, detecting a plurality of bounding boxes within the image, grouping the plurality of bounding boxes into a plurality of groups such that bounding boxes within a group have shared areas exceeding an area threshold, deriving a first subset of the plurality of bounding boxes by selecting bounding boxes having highest class confidence scores from at least one group, selecting a bounding box from the first subset having a highest score based on area and class confidence score, and outputting the selected bounding box.

First claim

What is claimed is:

1. A system for preprocessing an image for object recognition, the system comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to perform operations comprising:
receiving the image,
detecting a plurality of bounding boxes within the image,
grouping the plurality of bounding boxes into a plurality of groups, wherein bounding boxes within a group have shared areas exceeding an area threshold,
deriving a first subset of the plurality of bounding boxes by selecting bounding boxes having a highest class confidence scores from at least one group,
selecting a bounding box from the first subset having a highest score based on area and class confidence score, and
outputting the selected bounding box.

2. The system of claim 1 , wherein the operations further comprise paring the first subset by eliminating bounding boxes that do not include a center point of the image.

3. The system of claim 2 , wherein the selected bounding box is selected from the pared first subset.

4. The system of claim 1 , wherein the operations further comprise deriving a second subset of the plurality of bounding boxes by eliminating bounding boxes having one or more class confidence scores below a first class confidence score threshold.

5. The system of claim 4 , wherein each bounding box has a plurality of class confidence scores, and the second subset is derived by eliminating bounding boxes for which each class confidence score is below the first class confidence score threshold.

6. The system of claim 4 , wherein the first subset is derived from the second subset.

7. The system of claim 1 , wherein the operations further comprise deriving a second subset of the plurality of bounding boxes by eliminating bounding boxes having a highest class confidence score not within a selected class.

8. The system of claim 1 , wherein the operations further comprise cropping the image based on the selected bounding box.

9. The system of claim 8 , wherein cropping the image comprises cropping the image to include pixels enclosed by the selected bounding box and pixels located within a threshold of the selected bounding box.

10. The system of claim 9 , wherein the threshold comprises 10% of a corresponding dimension of the bounding box.

11. A system for preprocessing an image for object recognition, the system comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to perform operations comprising:
receiving the image having dimensions,
resizing the image to match input dimensions of a model,
using the model, detecting a plurality of bounding boxes within the resized image,
grouping the plurality of bounding boxes into a plurality of groups, wherein bounding boxes within a group have shared areas exceeding an area threshold,
deriving a first subset of the plurality of bounding boxes by selecting bounding boxes having a highest class confidence scores from at least one group,
selecting a bounding box from the first subset having a highest score based on area and class confidence score,
scaling the selected bounding box based on the dimensions of the received image and the input dimensions, and
outputting the scaled bounding box.

12. The system of claim 11 , further comprising a bus for receiving the image from the at least one memory.

13. The system of claim 11 , further comprising an image sensor for capturing the image and a bus for receiving the image from the image sensor.

14. The system of claim 11 , further comprising a network interface controller for receiving the image over a computer network.

15. The system of claim 11 , wherein the operations further comprise cropping the image based on the scaled bounding box.

16. The system of claim 15 , wherein cropping the image comprises overlaying the scaled bounding box on the image and cropping the image to include pixels enclosed by the overlaid bounding box and pixels located within a threshold of the overlaid bounding box.

17. The system of claim 16 , wherein the threshold comprises 10% of a corresponding dimension of the bounding box.

18. The system of claim 15 , wherein the cropped image is input to an object recognition algorithm.

19. The system of claim 18 , wherein the object recognition algorithm comprises a convolutional neural network.

20. A system for preprocessing an image for object recognition, the system comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to perform operations comprising:
receiving the image having dimensions,
resizing the image to match input dimensions of a model,
using the model, detecting a plurality of bounding boxes within the resized image,
deriving a first subset of the plurality of bounding boxes by eliminating bounding boxes having a highest class confidence score below a first class confidence score threshold,
deriving a second subset of the first subset by eliminating bounding boxes having a highest class confidence score not within a selected class,
grouping the bounding boxes of the second subset into a plurality of groups, wherein bounding boxes within a group have shared areas exceeding an area threshold,
deriving a third subset of the grouped bounding boxes by selecting a bounding box having a highest class confidence score from at least one group,
paring the third subset by eliminating bounding boxes that do not include a center point of the image,
selecting a bounding box from the pared third subset, the selected bounding box having a highest score based on area and class confidence score,
scaling the selected bounding box based on the dimensions of the received image and the input dimensions, and
outputting the scaled bounding box.

Assignees

• Capital One Services Llc

Inventors

• Tang Qiaochu
• Vasisht Sunil Subrahmanyam
• Wylie Stephen Michael
• Dagley Geoffrey
• Price Micah
• Hoover Jason Richard

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2115

  by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination · CPC title

• G06F18/24323

  Tree-organised classifiers · CPC title

• G06F18/24133

  Distances to prototypes · CPC title