Method for determining correct scanning distance using augmented reality and machine learning models

Patent metadata

Field	Value
Publication number	US-2022165070-A1
Application number	US-202217666256-A
Country	US
Kind code	A1
Filing date	Feb 7, 2022
Priority date	Oct 23, 2018
Publication date	May 26, 2022
Grant date	—

Abstract

A smart device is provided with an application program for displaying a video feed received from the smart device's camera. The application can determine the coordinates for an intersection point, which is a point on the ground where the smart device is pointing at. The application can display a target on the visual representation of the intersection point. Based on whether the smart device is at an appropriate distance from the intersection point, the user interface can superimpose an indicator on the video feed received from the camera. This can inform the user whether the smart device is at an optimal scan distance from the intersection point (or an object) so that the object can be identified by a machine learning model.

First claim

1 - 40 . (canceled)

41 . A method, comprising:
displaying a video feed, wherein the video feed is captured from a first position; determining an intersection point between a ground plane and the first position; determining a distance between the first position and the intersection point; displaying an indicator in a user interface, wherein a characteristic of the indicator is determined based on the distance between the first position and the intersection point; and capturing an image of an object in the video feed for use by an object recognition model in identifying the object when the distance between the first position and the intersection point is within an optimal scan distance.

42 . The method of claim 41 , wherein the optimal scanning distance is determined based on a threshold percentage of a field of view being occupied by the object.

43 . The method of claim 41 , further comprising determining the optimal scanning distance by:
capturing a first distance detection image of the object and a second distance detection image of the object; providing the two first distance detection image of the object and the second distance detection image of the object to the objection recognition model; receiving, from the object recognition model for the first distance detection image of the object, a first identity of the object and a first probability; receiving, from the objection recognition model for the second distance detection image of the object, a second identity of the object and a second probability; and determining the optimal scanning distance if the first probability and the second probability exceed a detection threshold probability.

44 . The method of claim 43 , wherein:
the first distance detection image and the second distance detection image are captured at different distances, and the method further comprises:
determining that the second probability is higher than the first probability, and
displaying a indication in a direction related to the second distance detection image.

45 . The method of claim 41 , wherein the indicator is displayed at a point over the video corresponding to the intersection point.

46 . The method of claim 45 , wherein the indicator is displayed over a visual representation of the intersection point.

47 . The method of claim 46 , wherein the indicator is displayed within the same plane as a visual representation of the ground plane.

48 . The method of claim 47 , wherein the optimal scan distance is determined based on an altitude of the first position relative to the ground plane.

49 . The method of claim 41 , further comprising:
identifying, using the object recognition model, an object shown in the video feed; and displaying information about the identified object.

50 . The method of claim 49 , wherein:
the object recognition model identifies the identified object as a vehicle, and the information displayed about the identified object comprises at least one of a vehicle make, a vehicle model, or a vehicle year.

51 . The method of claim 50 , further comprising:
conducting a search query for the vehicle; and displaying an inventory listing from a vehicle dealership.

52 . A device, comprising:
an image capture element configured to capture a video feed from a first position; a display configured to display the video feed; and a processor, wherein the processor is configured to:
determine an intersection point between a ground plane and the first position, and
determine a distance between the first position and the intersection point,
wherein the display is configured to display an indicator including a characteristic determined based on the distance between the first position and the intersection point, and wherein the image capture element is configured to capture an image of an object in the video feed for use by an object recognition model in identifying the object when the distance between the first position and the intersection point is within an optimal scan distance.

53 . The device of claim 52 , wherein the optimal scan distance is determined based on the size of an average object to be identified.

54 . The device of claim 52 , wherein the optimal scanning distance is determined based on at least one of the type of the device or the field of view of the image capture element.

55 . The device of claim 52 , wherein:
the characteristic of the indicator is a color, and the color changes based on the distance of the device from the intersection point.

56 . The device of claim 52 , wherein:
the characteristic of the indicator is a shape, and the shape changes based on the distance of the device from the intersection point.

57 . The device of claim 52 , wherein the indicator pulses if the distance between the device and the intersection point changes.

58 . A non-transitory computer-accessible medium having stored thereon computer-executable instructions executable by a computing hardware arrangement, wherein, when the computing hardware arrangement executes the instructions, the computing hardware arrangement is configured to perform procedures comprising:
displaying a video feed, wherein the video feed is captured from a first position; determining an intersection point between a ground plane and the first position; determining a distance between the first position and the intersection point; displaying an indicator in a user interface, wherein a characteristic of the indicator is determined based on the distance between the first position and the intersection point; and capturing an image of an object in the video feed for use by an object recognition model in identifying the object when the distance between the first position and the intersection point is within an optimal scan distance.

59 . The non-transitory computer-accessible medium of claim 58 , wherein the indicator is displayed at a point over the video corresponding to the intersection point.

60 . The non-transitory computer-accessible medium of claim 58 , wherein the optimal scan distance is a learned value received from the object recognition model.

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Castaneda Jonathan
• Lee Susie

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V10/17

  using hand-held instruments · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/584 Primary

  of vehicle lights or traffic lights · CPC title