Method for determining correct scanning distance using augmented reality and machine learning models

Patent metadata

Field	Value
Publication number	US-11594045-B2
Application number	US-202217666256-A
Country	US
Kind code	B2
Filing date	Feb 7, 2022
Priority date	Oct 23, 2018
Publication date	Feb 28, 2023
Grant date	Feb 28, 2023

Abstract

A smart device is provided with an application program for displaying a video feed received from the smart device's camera. The application can determine the coordinates for an intersection point, which is a point on the ground where the smart device is pointing at. The application can display a target on the visual representation of the intersection point. Based on whether the smart device is at an appropriate distance from the intersection point, the user interface can superimpose an indicator on the video feed received from the camera. This can inform the user whether the smart device is at an optimal scan distance from the intersection point (or an object) so that the object can be identified by a machine learning model.

First claim

The invention claimed is:

1. A method, comprising:
displaying on a display a video feed, wherein the video feed is captured by a camera from a first position;
determining an intersection point between a ground plane and the first position;
determining a distance between the first position and the intersection point;
displaying on a display an indicator in a user interface, wherein a characteristic of the indicator is determined based on the distance between the first position and the intersection point; and
capturing by a camera an image of an object in the video feed for use by an object recognition model in identifying the object within the captured image when the distance between the first position and the intersection point is within an optimal scan distance.

2. The method of claim 1 , wherein the optimal scanning distance is determined based on a threshold percentage of a field of view being occupied by the object.

3. The method of claim 1 , further comprising determining the optimal scanning distance by:
capturing a first distance detection image of the object and a second distance detection image of the object;
providing the two first distance detection image of the object and the second distance detection image of the object to the objection recognition model;
receiving, from the object recognition model for the first distance detection image of the object, a first identity of the object and a first probability;
receiving, from the objection recognition model for the second distance detection image of the object, a second identity of the object and a second probability; and
determining the optimal scanning distance if the first probability and the second probability exceed a detection threshold probability.

4. The method of claim 3 , wherein:
the first distance detection image and the second distance detection image are captured at different distances, and
the method further comprises:
determining that the second probability is higher than the first probability, and
displaying a indication in a direction related to the second distance detection image.

5. The method of claim 1 , wherein the indicator is displayed at a point over the video corresponding to the intersection point.

6. The method of claim 5 , wherein the indicator is displayed over a visual representation of the intersection point.

7. The method of claim 6 , wherein the indicator is displayed within the same plane as a visual representation of the ground plane.

8. The method of claim 7 , wherein the optimal scan distance is determined based on an altitude of the first position relative to the ground plane.

9. The method of claim 1 , further comprising:
identifying, using the object recognition model, an object shown in the video feed; and
displaying information about the identified object.

10. The method of claim 9 , wherein:
the object recognition model identifies the identified object as a vehicle, and
the information displayed about the identified object comprises at least one of a vehicle make, a vehicle model, or a vehicle year.

11. The method of claim 10 , further comprising:
conducting a search query for the vehicle; and
displaying an inventory listing from a vehicle dealership.

12. A device, comprising:
a camera configured to capture a video feed from a first position;
a display configured to display the video feed; and
a processor,
wherein the processor is configured to:
determine an intersection point between a ground plane and the first position, and
determine a distance between the first position and the intersection point,

wherein the display is configured to display an indicator including a characteristic determined based on the distance between the first position and the intersection point, and
wherein the camera is configured to capture an image of an object in the video feed for use by an object recognition model in identifying the object within the captured image when the distance between the first position and the intersection point is within an optimal scan distance.

13. The device of claim 12 , wherein the optimal scan distance is determined based on the size of an average object to be identified.

14. The device of claim 12 , wherein the optimal scanning distance is determined based on at least one of the type of the device or the field of view of the camera.

15. The device of claim 12 , wherein:
the characteristic of the indicator is a color, and
the color changes based on the distance of the device from the intersection point.

16. The device of claim 12 , wherein:
the characteristic of the indicator is a shape, and
the shape changes based on the distance of the device from the intersection point.

17. The device of claim 12 , wherein the indicator pulses if the distance between the device and the intersection point changes.

18. A non-transitory computer-accessible medium having stored thereon computer-executable instructions executable by a computing hardware arrangement, wherein, when the computing hardware arrangement executes the instructions, the computing hardware arrangement is configured to perform procedures comprising:
displaying on a display a video feed, wherein the video feed is captured by a camera from a first position;
determining an intersection point between a ground plane and the first position;
determining a distance between the first position and the intersection point;
displaying on a display an indicator in a user interface, wherein a characteristic of the indicator is determined based on the distance between the first position and the intersection point; and
capturing by the camera an image of an object in the video feed for use by an object recognition model in identifying the object within the captured image when the distance between the first position and the intersection point is within an optimal scan distance.

19. The non-transitory computer-accessible medium of claim 18 , wherein the indicator is displayed at a point over the video corresponding to the intersection point.

20. The non-transitory computer-accessible medium of claim 18 , wherein the optimal scan distance is a learned value received from the object recognition model.

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Castaneda Jonathan
• Lee Susie

Classifications

• G01B11/03

  by measuring coordinates of points · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title