Method for determining correct scanning distance using augmented reality and machine learning models

Patent metadata

Field	Value
Publication number	US-10521681-B1
Application number	US-201916456108-A
Country	US
Kind code	B1
Filing date	Jun 28, 2019
Priority date	Oct 23, 2018
Publication date	Dec 31, 2019
Grant date	Dec 31, 2019

Abstract

A smart device is provided with an application program for displaying a video feed received from the smart device's camera. The application can determine the coordinates for an intersection point, which is a point on the ground where the smart device is pointing at. The application can display a target on the visual representation of the intersection point. Based on whether the smart device is at an appropriate distance from the intersection point, the user interface can superimpose an indicator on the video feed received from the camera. This can inform the user whether the smart device is at an optimal scan distance from the intersection point (or an object) so that the object can be identified by a machine learning model.

First claim

The invention claimed is:

1. A method comprising:
displaying a user interface on a smart device, wherein the user interface is configured to display a video feed received from an image capture element of the smart device;
detecting a ground plane in the video feed;
displaying a target over a point in the video feed, wherein the target corresponds to an intersection point where the plane at which the smart device is directed intersects with the ground plane;
determining a distance between the smart device and the intersection point;
comparing the distance with a threshold distance value, wherein the threshold distance value is calculated by determining a probability value for identification of at least one object;
displaying an indicator in the user interface, wherein the indicator reflects the comparison of the distance with the threshold distance value;
capturing image data of the at least one object;
providing the image data to a machine learning model to identify information about the at least one object in the image data; and
displaying the information about the at least one object in the user interface of the smart device.

2. The method of claim 1 , wherein the intersection point corresponds to a centerpoint of a field of view of the image capture element of the smart device.

3. The method of claim 1 , wherein the smart device is configured to display the indicator coplanar with the ground plane.

4. The method of claim 1 , wherein the indicator is indicative of a target distance between the smart device and the at least one object.

5. The method of claim 1 , wherein:
the indicator changes color based on the distance of the smart device from the intersection point,
the indicator pulses if the distance between the smart device and the intersection point is more than the threshold distance value, or
a shape of the indicator changes based on the distance of the smart device from the intersection point.

6. The method of claim 1 , wherein the capturing image data of the at least one object occurs:
when the indicator indicates the distance is within the threshold distance value, or
in response to receiving a command from a user of the smart device through the user interface.

7. A device comprising:
a processor, a memory, a display and an image capture element, wherein:
the display is configured to show a user interface to display a video feed received from the image capture element of the device;
the processor is configured to detect a ground plane in the video feed;
the display is configured to show a target over a point in the video feed, wherein the target corresponds to an intersection point where the plane at which the device is directed intersects with the ground plane;
the processor is configured to determine a distance between the device and the intersection point;
the processor is configured to compare the distance with a threshold distance value, wherein the threshold distance value is calculated by determining a probability value for identification of at least one object;
the display is configured to show an indicator in the user interface, wherein the indicator reflects the comparison of the distance with the threshold distance value;
the image capture element is configured to capture image data of the at least one object;
the processor is configured to provide the image data to a machine learning model to identify information about the at least one object in the image data; and
the display is configured to show the information about the at least one object in the user interface of the device.

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Castaneda Jonathan
• Lee Susie

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V10/17

  using hand-held instruments · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/584

  of vehicle lights or traffic lights · CPC title