Method for determining correct scanning distance using augmented reality and machine learning models

Patent metadata

Field	Value
Publication number	US-12444147-B2
Application number	US-202418655505-A
Country	US
Kind code	B2
Filing date	May 6, 2024
Priority date	Oct 23, 2018
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

A smart device is provided with an application program for displaying a video feed received from the smart device's camera. The application can determine the coordinates for an intersection point, which is a point on the ground where the smart device is pointing at. The application can display a target on the visual representation of the intersection point. Based on whether the smart device is at an appropriate distance from the intersection point, the user interface can superimpose an indicator on the video feed received from the camera. This can inform the user whether the smart device is at an optimal scan distance from the intersection point (or an object) so that the object can be identified by a machine learning model.

First claim

The invention claimed is:

1. A method comprising:
determining, on a display of a video feed from a camera of a smart device, an intersection between a ground plane and the camera;
overlaying, on the video feed display of the smart device, a reticle comprising at least one target ellipse on the determined intersection between the ground plane and the camera;
identifying, using an object recognition model, an object shown in the video feed; and
displaying information about the identified object when the camera is oriented such that the identified object is within the reticle.

2. The method of claim 1 , wherein the determining of the intersection between the ground plane and the camera includes feature extraction through visual processing.

3. The method of claim 1 , wherein the smart device is configured to determine coordinates of the smart device and coordinates of the intersection in a 3D space.

4. The method of claim 3 , wherein a distance is determined based on the determined coordinates of the smart device and the intersection.

5. The method of claim 4 , wherein the target ellipse is determined based on a comparison of the determined distance with a threshold distance value.

6. The method of claim 1 , wherein in response to the smart device moving away from the intersection point, the user interface displays more ellipses.

7. The method of claim 1 , wherein in response to the smart device approaching the intersection point, the user interface displays fewer ellipses.

8. The method of claim 1 , wherein an optimal scanning distance is determined based on a threshold percentage of a field of view being occupied by the object.

9. The method of claim 1 , further comprising determining an optimal scanning distance by:
capturing a first distance detection image of the object and a second distance detection image of the object;
providing the two first distance detection image of the object and the second distance detection image of the object to the objection recognition model;
receiving, from the object recognition model for the first distance detection image of the object, a first identity of the object and a first probability;
receiving, from the objection recognition model for the second distance detection image of the object, a second identity of the object and a second probability; and
determining the optimal scanning distance if the first probability and the second probability exceed a detection threshold probability.

10. The method of claim 9 , wherein:
the first distance detection image and the second distance detection image are captured at different distances, and
the method further comprises:
determining that the second probability is higher than the first probability, and
displaying an indication in a direction related to the second distance detection image.

11. A device comprising:
a smart device including a display, a camera, and at least one processor, the at least one processor configured to:
determine, on the display of a video feed from the camera, an intersection between a ground plane and the camera;
overlay, on the video feed display of the smart device, a reticle comprising at least one target ellipse on the determined intersection between the ground plane and the camera;
identify, using an object recognition model, an object shown in the video feed; and
show, on the display, information about the identified object when the camera is oriented such that the identified object is within the reticle.

12. The device of claim 11 , wherein the smart device is configured to determine coordinates of the smart device and coordinates of the intersection in a 3D space.

13. The device of claim 12 , wherein a distance is determined based on the determined coordinates of the smart device and the intersection.

14. The device of claim 13 , wherein the target ellipse is determined based on a comparison of the determined distance with a threshold distance value.

15. The device of claim 11 , wherein in response to the smart device moving away from the intersection point, the user interface displays more ellipses.

16. The device of claim 11 , wherein in response to the smart device approaching the intersection point, the user interface displays fewer ellipses.

17. The device of claim 11 , wherein an optimal scanning distance is determined based on a threshold percentage of a field of view being occupied by the object.

18. The device of claim 11 , further comprising determining an optimal scanning distance by:
capturing a first distance detection image of the object and a second distance detection image of the object;
providing the two first distance detection image of the object and the second distance detection image of the object to the objection recognition model;
receiving, from the object recognition model for the first distance detection image of the object, a first identity of the object and a first probability;
receiving, from the objection recognition model for the second distance detection image of the object, a second identity of the object and a second probability; and
determining the optimal scanning distance if the first probability and the second probability exceed a detection threshold probability.

19. The device of claim 18 , wherein:
the first distance detection image and the second distance detection image are captured at different distances, and
the method further comprises:
determining that the second probability is higher than the first probability, and
displaying an indication in a direction related to the second distance detection image.

20. A non-transitory computer-accessible medium having stored thereon computer-executable instructions executable by a computing hardware arrangement, wherein, when the computing hardware arrangement executes the instructions, the computing hardware arrangement is configured to perform procedures comprising:
determining, on a display of a video feed from a camera of a smart device, an intersection between a ground plane and the camera;
overlaying, on the video feed display of the smart device, a reticle comprising at least one target ellipse on the determined intersection between the ground plane and the camera;
identifying, using an object recognition model, an object shown in the video feed; and
displaying information about the identified object when the camera is oriented such that the identified object is within the reticle.

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Castaneda Jonathan
• Lee Susie

Classifications

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/17

  using hand-held instruments · CPC title

• G06V20/584

  of vehicle lights or traffic lights · CPC title