Image analysis and processing pipeline with real-time feedback and autocapture capabilities, and visualization and configuration system

Patent metadata

Field	Value
Publication number	US-10691966-B2
Application number	US-201916369807-A
Country	US
Kind code	B2
Filing date	Mar 29, 2019
Priority date	Aug 20, 2018
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

A device may capture a plurality of preview frames of a document, and for each preview frame of the plurality of preview frames, process the preview frame to identify an object in the preview frame. Processing the preview frame may include converting the preview frame into a grayscale image, generating a blurred image based on the grayscale image, detecting a plurality of edges in the blurred image, defining at least one bounding rectangle based on the plurality of edges, and determining an outline of the object based on the at least one bounding rectangle. The device may determine whether a value of an image parameter, associated with the one or more preview frames, satisfies a threshold, and provide feedback to a user of the device, or automatically capture an image of the document, based on determining whether the value of the image parameter satisfies the threshold.

First claim

What is claimed is:

1. A method, comprising:
processing, by a device, a preview frame to identify an object,
the preview frame being obtained from a camera of the device and provided for display;

determining, by the device and based on identifying the object, whether a value of an image parameter, associated with the preview frame, satisfies a first threshold,
the image parameter relating to a distance between the device and the object;

applying, by the device, a filter to the preview frame for glare detection;
determining, by the device and based on applying the filter, whether a value relating to glare satisfies a second threshold; and
selectively:
providing, by the device, first feedback based on determining that the value of the image parameter does not satisfy the first threshold,
the first feedback including an instruction to perform an action with respect to at least one of the device or the object;

providing, by the device, second feedback based on determining that the value relating to glare does not satisfy the second threshold; or
automatically capturing, by the device, an image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold.

2. The method of claim 1 , where the preview frame is one of a plurality of preview frames obtained in real-time or near real-time.

3. The method of claim 1 , where processing the preview frame comprises:
converting the preview frame into a grayscale image.

4. The method of claim 3 , where processing the preview frame further comprises at least one of:
blurring the grayscale image,
applying another filter to the grayscale image,
detecting shapes in the preview frame, or
defining boundaries in the preview frame.

5. The method of claim 1 , further comprising:
determining whether a value relating to contrast satisfies a third threshold; and
providing third feedback based on determining that the value relating to contrast does not satisfy the third threshold.

6. The method of claim 1 , further comprising:
determining an outline of the object; and
causing a boundary box, corresponding to the outline, to be displayed.

7. The method of claim 1 , further comprising:
determining an outline of the object;
causing a boundary box, corresponding to the outline, to be displayed; and
periodically updating, based on movements of the device, at least one of:
a position of the boundary box,
an orientation of the boundary box, or
a size of the boundary box.

8. A device, comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, configured to:
process a preview frame to identify an object,
the preview frame being obtained from a camera of the device and provided for display;

determine, based on identifying the object, whether a value of an image parameter, associated with the preview frame, satisfies a first threshold,
the image parameter relating to a distance between the device and the object;

apply a filter to the preview frame for glare detection;
determine, based on applying the filter, whether a value relating to glare satisfies a second threshold; and
selectively:
provide first feedback based on determining that the value of the image parameter does not satisfy the first threshold,
the first feedback including an instruction to perform an action with respect to at least one of the device or the object;

provide second feedback based on determining that the value relating to glare does not satisfy the second threshold; or
automatically capture an image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold.

9. The device of claim 8 , where the value of the image parameter is a ratio based on a width of an outline associated with the object, and a width of a display.

10. The device of claim 8 , where the one or more processors, when providing the first feedback, are to:
provide the first feedback in real-time or near-real time.

11. The device of claim 8 , where the one or more processors, when automatically capturing the image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold, are to:
automatically capture the image of the object based on determining that multiple values, associated with image parameters for multiple preview frames, satisfy corresponding first thresholds and based on determining that the value relating to glare satisfies the second threshold.

12. The device of claim 8 , where the value of the image parameter is a first value of the image parameter;
where the preview frame is a first preview frame; and
where the one or more processors are further to:
perform an analysis of the image based on capturing the image;
determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and
process a second preview frame based on determining that the second value of the image parameter does not satisfy the first threshold.

13. The device of claim 8 , where the value of the image parameter is a first value of the image parameter; and
where the one or more processors are further to:
perform an analysis of the image based on capturing the image;
determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and
crop, based on determining that the second value of the image parameter satisfies the first threshold, the object from the image.

14. The device of claim 8 , where the value of the image parameter is a first value of the image parameter; and
where the one or more processors are further to:
perform an analysis of the image based on capturing the image;
determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and
provide, based on determining that the second value of the image parameter satisfies the first threshold, the image to a server.

15. A non-transitory computer-readable medium storing instructions, the instructions comprising:
one or more instructions that, when executed by one or more processors, cause the one or more processors to:
process a preview frame to identify an object,
the preview frame being obtained from a camera of a device and provided for display;

determine, based on identifying the object, whether a value of an image parameter, associated with the preview frame, satisfies a first threshold,
the image parameter relating to a distance between the device and the object;

apply a filter to the preview frame for glare detection;
determine, based on applying the filter, whether a value relating to glare satisfies a second threshold; and
selectively:
provide first feedback based on determining that the value of the image parameter does not satisfy the first threshold,
the first feedback including an instruction to perform an action with respect to at least one of the device or the object;

provide second feedback based on determining that the value relating to glare does not satisfy the second threshold; or
automatically capture an image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold

Assignees

• Capital One Services Llc

Inventors

• Pribble Jason
• Jarvis Daniel Alan
• Capurso Nicholas
• Cerney Eric

Classifications

• G06V10/993 Primary

  Evaluation of the quality of the acquired pattern · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06V10/242

  by image rotation, e.g. by 90 degrees · CPC title

• G06V10/141

  Control of illumination · CPC title

• G06V10/30

  Noise filtering · CPC title