Systems and methods for automatic image capture on a mobile device

Patent metadata

Field	Value
Publication number	US-11539848-B2
Application number	US-202016889131-A
Country	US
Kind code	B2
Filing date	Jun 1, 2020
Priority date	Jan 18, 2008
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

Real-time evaluation and enhancement of image quality prior to capturing an image of a document on a mobile device is provided. An image capture process is initiated on a mobile device during which a user of the mobile device prepares to capture the image of the document, utilizing hardware and software on the mobile device to measure and achieve optimal parameters for image capture. Feedback may be provided to a user of the mobile device to instruct the user on how to manually optimize certain parameters relating to image quality, such as the angle, motion and distance of the mobile device from the document. When the optimal parameters for image capture of the document are achieved, at least one image of the document is automatically captured by the mobile device.

First claim

What is claimed is:

1. A method of capturing an image of a document on a mobile device, the method comprising using at least one hardware processor of the mobile device to:
initiate a video mode of an image capture application, on the mobile device, that begins capturing a video stream via a camera of the mobile device;
continuously display video frames from the video stream on a display of the mobile device;
for each of a plurality of the video frames from the video stream, measure at least one parameter representing an image quality of the video frame, in real time, to detect a video frame of a document for which the at least one parameter exceeds a threshold;
detect the video frame of the document for which the at least one parameter exceeds the threshold; and,
in response to detecting the video frame of the document for which the at least one parameter exceeds the threshold, automatically, without a user having to perform any input to the mobile device, both
select at least one of the video frames for a final image of the document, and
stop capture of the video stream.

2. The method of claim 1 , wherein selecting at least one of the video frames comprises:
processing a plurality of video frames in close proximity to the detected video frame; and
selecting the at least one video frame from among the detected video frame and the plurality of video frames in close proximity to the detected video frame that has a highest image quality.

3. The method of claim 2 , wherein selecting the at least one video frame from among the detected video frame and the plurality of video frames in close proximity to the detected video frame comprises creating a composite image of the document by combining a video frame that has a highest image quality with one or more adjacent video frames.

4. The method of claim 1 , further comprising using the at least one hardware processor of the mobile device to, while continuously displaying the video frames from the video stream on the display of the mobile device, display real-time visual feedback to the user that indicates how to adjust the capture of the video stream relative to the document.

5. The method of claim 4 , wherein the visual feedback comprises a frame corresponding to edges of the document.

6. The method of claim 5 , further comprising using the at least one hardware processor to change a color of the frame when the document is properly positioned within the frame.

7. The method of claim 4 , wherein the visual feedback comprises one or more messages.

8. The method of claim 1 , further comprising using the at least one hardware processor to, after selecting the at least one video frame for the final image of the document, generate the final image of the document by at least cropping the at least one video frame using edge detection to detect edges of the document in the at least one video frame.

9. The method of claim 8 , wherein generating the final image of the document further comprises converting the at least one video frame to grayscale.

10. The method of claim 1 , further comprising using the at least one hardware processor to upload the final image of the document to a remote server via at least one network.

11. The method of claim 10 , wherein the document is a check.

12. The method of claim 1 , wherein the at least one parameter comprises one or more of focus, exposure, contrast, or angle.

13. The method of claim 1 , wherein the at least one parameter comprises a plurality of parameters, and wherein detecting a video frame of a document for which the at least one parameter exceeds a threshold comprises, for each video frame:
combining measurements of the plurality of parameters for the video frame into an overall quality score; and
comparing the overall quality score to the threshold.

14. The method of claim 1 , wherein the at least one parameter comprises a plurality of parameters, and wherein detecting a video frame of a document for which the at least one parameter exceeds a threshold comprises, for each video frame, comparing each of the plurality of parameters for the video frame to a respective threshold.

15. The method of claim 1 , further comprising using the at least one hardware processor to, during capture of the video stream, change the at least one parameter by adjusting one or more features of the camera.

16. The method of claim 15 , wherein the one or more features comprise one or more of focus, contrast, shutter period, or aperture.

17. The method of claim 1 , further comprising using the at least one hardware processor to adjust the threshold when, after a period of time, no video frame is detected for which the at least one parameter exceeds the threshold.

18. The method of claim 1 , further comprising using the at least one hardware processor to, when, after a period of time, no video frame is detected for which the at least one parameter exceeds the threshold, provide control of the camera to the user with instructions to capture the final image of the document manually.

19. A system comprising:
a camera;
a display;
at least one hardware processor; and
one or more software modules that are configured to, when executed by the at least one hardware processor,
initiate a video mode of an image capture application that begins capturing a video stream via the camera,
continuously display video frames from the video stream on the display,
for each of a plurality of the video frames from the video stream, measure at least one parameter representing an image quality of the video frame, in real time, to detect a video frame of a document for which the at least one parameter exceeds a threshold,
detect the video frame of the document for which the at least one parameter exceeds the threshold, and,
in response to detecting the video frame of the document for which the at least one parameter exceeds the threshold, automatically, without a user having to perform any input, both select at least one of the video frames for a final image of the document, and stop capture of the video stream.

20. A non-transitory computer-readable medium having instructions stored therein, wherein the instructions, when executed by a processor of a mobile device, cause the processor to:
initiate a video mode of an image capture application, on the mobile device, that begins capturing a video stream via a camera of the mobile device;
continuously display video frames from the video stream on a display of the mobile device;
for each of a plurality of the video frames from the video stream, measure at least one parameter representing an image quality of the video frame, in real time, to detect a video frame of a document for which the at least one parameter exceeds a threshold;
detect the video frame of the document for which the at least one parameter exceeds the threshold; and,
in response to detecting the video frame of the document for which the at least one parameter exceeds the threshold, automatically, without a user having to perform any input to the mobile device, both
select at least one of the video frames for a final image of the document, and
stop capture of the video stream.

Assignees

• Mitek Systems Inc

Inventors

• Roach John J
• Nepomniachtchi Grigori
• Couch Robert
• Rybakov Oleg
• Gillen Michael
• Bell Kevin Andrew

Classifications

• H04N1/00129 Primary

  with a display device, e.g. CRT or LCD monitor (with a television apparatus H04N1/00283; arrangements for control of display devices G09G) · CPC title

• G06V30/1475 Primary

  Inclination or skew detection or correction of characters or of image to be recognised · CPC title

• G06V30/166

  Normalisation of pattern dimensions · CPC title

• G06V30/162

  Quantising the image signal · CPC title

• G06Q20/3276

  using a pictured code, e.g. barcode or QR-code, being read by the M-device · CPC title