Method for determining correct scanning distance using augmented reality and machine learning models

Patent metadata

Field	Value
Publication number	US-2020250444-A1
Application number	US-202016858173-A
Country	US
Kind code	A1
Filing date	Apr 24, 2020
Priority date	Oct 23, 2018
Publication date	Aug 6, 2020
Grant date	—

Abstract

A smart device is provided with an application program for displaying a video feed received from the smart device's camera. The application can determine the coordinates for an intersection point, which is a point on the ground where the smart device is pointing at. The application can display a target on the visual representation of the intersection point. Based on whether the smart device is at an appropriate distance from the intersection point, the user interface can superimpose an indicator on the video feed received from the camera. This can inform the user whether the smart device is at an optimal scan distance from the intersection point (or an object) so that the object can be identified by a machine learning model.

First claim

1 - 20 . (canceled)

21 . A method comprising:
displaying a video feed, wherein the video feed is captured from a first position; determining an intersection point between a ground plane and the first position; determining a distance between the first position and the intersection point; displaying an indicator in the user interface, wherein a characteristic of the indicator is determined based on the distance between the first position and the intersection point; identifying, using an object recognition model, an object shown in the video feed; and displaying the information about the object.

22 . The method of claim 21 , wherein the intersection point is a point on a ground plane where a perpendicular hypothetical line extending from the first position would intersect the ground plane.

23 . The method of claim 22 , wherein the intersection point corresponds to a centerpoint of a field of view of a camera located at the first position.

24 . The method of claim 21 , wherein the indicator is displayed at a point over the video corresponding to the intersection point.

25 . The method of claim 24 , wherein the indicator is displayed over a visual representation of the intersection point.

26 . The method of claim 25 , wherein the indicator is displayed within the same plane as a visual representation of the ground plane.

27 . The method of claim 21 , further comprising capturing an image of the object for use by the object recognition model in identifying the object.

28 . The method of claim 27 , wherein the image of the object is captured automatically based upon the distance between the first position and the intersection point.

29 . The method of claim 28 , wherein the image of the object is captured automatically when the distance between the first position and the intersection point is within an optimal scan distance.

30 . The method of claim 29 , wherein the optimal scan distance is determined based on an altitude of the first position relative to the ground plane.

31 . The method of claim 29 , wherein the optical scan distance is determined based on a field of view and an angular separation of pixels of a camera located at the first position.

32 . The method of claim 27 , wherein a notification is displayed in the video feed upon the capture of the image of the object.

33 . The method of claim 21 , wherein the user interface is configured to change the characteristic of the indicator when the distance between the first position and the intersection point changes.

34 . The method of claim 21 , further comprising:
comparing the distance between the first position and the intersection point with a threshold distance; and displaying the characteristic of the indicator based on the comparison.

35 . A non-transitory computer-accessible medium having stored thereon computer-executable instructions for an application for identifying objects executable by a computing hardware arrangement, wherein, when the computing hardware arrangement executes the instructions, the computing hardware arrangement is configured to perform procedures comprising:
displaying a video feed, wherein the video feed is captured from a first position; determining a distance between the first position and an intersection point; comparing the distance between the device and the intersection point with a threshold distance; and displaying an indicator, wherein the indicator is determined based on the comparison; identifying, using an object recognition model, a product shown in the video feed; and displaying information about the product.

36 . The non-transitory computer-accessible medium of claim 35 , wherein the intersection point is a point on a ground plane where a perpendicular hypothetical line extending from the first position would intersect the ground plane.

37 . The non-transitory computer-accessible medium of claim 35 , further comprising:
capturing a frame of the video feed when a condition is met; conducting a search query for the product identified in the frame; and displaying a listing including the product.

38 . The non-transitory computer-accessible medium of claim 37 , wherein the condition is met when the device is within an optimal scan distance of the product.

39 . The non-transitory computer-accessible medium of claim 38 , wherein the optimal scan distance is determined based on an altitude of the first position relative to the ground plane.

40 . The non-transitory computer-accessible medium of claim 38 , wherein the optical scan distance is determined based on a field of view and an angular separation of pixels of a camera located at the first position.

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Castaneda Jonathan
• Lee Susie

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V10/17

  using hand-held instruments · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/584

  of vehicle lights or traffic lights · CPC title