Automatic image cropping using a reference feature

Patent metadata

Field	Value
Publication number	US-12380593-B2
Application number	US-202218053576-A
Country	US
Kind code	B2
Filing date	Nov 8, 2022
Priority date	Nov 8, 2022
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

In some implementations, a device may receive the image of the document, the image of the document depicting the reference feature, and the reference feature being associated with one or more location parameters for a document type associated with the document. The device may detect a location of the reference feature as depicted in the image, the location being defined by bounds of the reference feature as depicted in the image. The device may detect a border of the document as depicted in the image based on identifying one or more edges of the document based on the bounds of the reference feature and the one or more location parameters. The device may modify the image of the document based on the border of the document to obtain a cropped image. The device may transmit, to a server device, the cropped image.

First claim

What is claimed is:

1. A system for automatic image cropping using a reference feature, the system comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, configured to:
obtain an image of a document, the image of the document depicting the reference feature, and the reference feature being associated with a defined location, size, and orientation for a document type associated with the document;
obtain, via an output of an object detection model, an indication of a location of the reference feature as depicted in the image, the indication of the location including a bounding box;
detect that the reference feature as depicted in the image is obfuscated based on comparing the bounding box to the defined location, size, and orientation;
perform, based on detecting that the reference feature as depicted in the image is obfuscated, an action to modify the bounding box;
detect an outline of the document as depicted in the image based on identifying one or more edges of the document by expanding the bounding box, as modified, based on the defined location, size, and orientation of the reference feature;
crop the image of the document based on the outline of the document to obtain a cropped image; and
provide, to a server device, the cropped image.

2. The system of claim 1 , wherein the one or more processors are further configured to:
process the image of the document to modify an orientation or a rotation of the image of the document to obtain an oriented image; and
provide the oriented image as an input to the object detection model, wherein the output of the object detection model is based on the oriented image.

3. The system of claim 1 , wherein the reference feature is a first reference feature and the location is a first location, and wherein the one or more processors are further configured to:
detect a second location of a second reference feature depicted in the image of the document;
compare the second location to the first location; and
determine whether the first location is correctly detected based on comparing the second location to the first location.

4. The system of claim 3 , wherein the one or more processors, to modify the bounding box, are further configured to:
modify the bounding box based on determining that the first location is not correctly detected, wherein the bounding box is modified based on the first location and a correlation between the first location and the defined location.

5. The system of claim 1 , wherein the one or more processors, to detect the outline of the document as depicted in the image, are configured to:
calculate a size of the document as depicted in the image;
determine a perspective distortion associated with the image; and
calculate, based on the size of the document, the perspective distortion, and the defined location, size, and orientation, at least one of:
a first distance from a top of the bounding box to a top edge of the document,
a second distance from a bottom of the bounding box to a bottom edge of the document,
a third distance from a right side of the bounding box to a right edge of the document, or
a fourth distance from a left side of the bounding box to a left edge of the document,
wherein the outline of the document is based on the top edge, the bottom edge, the right edge, or the left edge.

6. The system of claim 1 , wherein the one or more processors are further configured to:
trigger the image of the document to be stored based on detecting the outline of the document.

7. The system of claim 1 , wherein the reference feature includes at least one of:
a machine-readable zone,
a text field,
a user image, or
a barcode.

8. A method of automatically cropping an image of a document using a reference feature, comprising:
receiving, by a device, the image of the document, the image of the document depicting the reference feature, and the reference feature being associated with one or more location parameters for a document type associated with the document;
detecting, by the device, a location of the reference feature as depicted in the image, the location being defined by bounds of the reference feature as depicted in the image;
detecting that the reference feature as depicted in the image is obfuscated based on comparing the bounds of the reference feature to the one or more location parameters;
performing, based on detecting that the reference feature as depicted in the image is obfuscated, an action to modify the bounds of the reference feature;
detecting, by the device, a border of the document as depicted in the image based on identifying one or more edges of the document based on the bounds of the reference feature, as modified, and the one or more location parameters;
modifying, by the device, the image of the document based on the border of the document to obtain a cropped image; and
transmitting, by the device and to a server device, the cropped image.

9. The method of claim 8 , wherein the one or more location parameters indicate one or more percentage values indicating distances between one or more edges of the reference feature and the one or more edges of the document, and wherein detecting the border of the document comprises:
calculating a size of the reference feature as depicted in the image; and
calculating one or more distances between the bounds of the reference feature and the one or more edges of the document based on multiplying the one or more percentage values by the size of the reference feature.

10. The method of claim 8 , wherein detecting the location of the reference feature as depicted in the image comprises:
analyzing, using optical character recognition (OCR), the image to detect the bounds of the reference feature, the bounds of the reference feature being defined by a bounding box.

11. The method of claim 10 , wherein the one or more location parameters indicate a pattern between the location of the reference feature and the border of the document, and wherein detecting the border of the document as depicted in the image comprises:
projecting one or more edges of the bounding box based on the pattern and a size and an orientation of the image to detect one or more edges of the document that define the border.

12. The method of claim 8 , wherein detecting the border of the document as depicted in the image comprises:
detecting a perspective orientation associated with the image; and
modifying the bounds of the reference feature as depicted in the image based on the perspective orientation.

13. The method of claim 8 , wherein detecting the location of the reference feature comprises:
modifying the image to a top-down view of the image;
detecting the location of the reference feature using the top-down view of the image; and
inserting a bounding box defining the bounds of the reference feature, as detected using the top-down view, into the image of the document.

14. The method of claim 8 , further comprising:
performing perspective correction for the image based on the bounds of the reference feature as detected in the image.

15. A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the device to:
obtain an image of a document, the image of the document depicting a reference feature, and the reference feature being associated with one or more location parameters for a document type associated with the document;
obtain, via an output of an object detec

Assignees

• Capital One Services Llc

Inventors

• Pribble Jason
• Patil Swapnil
• Mcdonald Alexandria

Classifications

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06T2207/20132

  Image cropping · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V10/267

  by performing operations on regions, e.g. growing, shrinking or watersheds · CPC title

• G06T2207/30176

  Document · CPC title