Methods and systems for providing a tutorial for graphic manipulation of objects including real-time scanning in an augmented reality

Patent metadata

Field	Value
Publication number	US-11461980-B2
Application number	US-202016911803-A
Country	US
Kind code	B2
Filing date	Jun 25, 2020
Priority date	Jun 28, 2019
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

A device including a storage medium storing instructions, and at least one processor configured to execute the stored instructions to perform operations is provided. The operations include displaying on a display screen an interactive scene containing an augmented reality element, receiving input to change display of the interactive scene, and receiving a request to simulate the capturing of image data representing the interactive scene. The operations further include determining a position of the augmented reality element relative to a region defined on the display screen, simulating the capturing of the image data representing the interactive scene, and providing, based on the determined position, an indication of whether the simulation has succeeded.

First claim

What is claimed is:

1. A device, comprising:
a storage medium storing instructions; and
at least one processor configured to execute the stored instructions to perform operations comprising:
displaying on a display screen an interactive scene containing an augmented reality element and a region represented by a first graphical user interface element;
receiving an input from a user, the input being generated from a motion associated with positioning the display screen so as to place the augmented reality element at least partially within the region;
providing a first indication of whether a capturing of an object in a position of the augmented reality element may enable successful recognition of the object, the first indication based on determining a likelihood of successful recognition of the object by:
identifying a probability of determining an identity of the object based on a location of the object within the region;
determining a match score based on the probability; and
determining whether the match score is above a threshold value; and

displaying suggestions for manipulating the device based on the match score, the suggestions including virtually rotating the augmented reality element to simulate a repositioning of the device.

2. The device of claim 1 , wherein the input comprises at least one of:
zooming a viewpoint to the augmented reality element;
rotating the viewpoint for the interactive scene;
moving the viewpoint for the interactive scene; or
changing lighting conditions of the interactive scene.

3. The device of claim 1 , wherein the operations further comprise:
receiving a request to retrieve the augmented reality element from a database; and
retrieving the augmented reality element from the database.

4. The device of claim 1 , wherein displaying the interactive scene comprises displaying a view of an environment of the user.

5. The device of claim 1 , wherein providing the first indication comprises:
identifying a data record in a database corresponding to the object.

6. The device of claim 1 , wherein the augmented reality element corresponds to a virtual vehicle.

7. The device of claim 1 , wherein the operations further comprise:
initially displaying the augmented reality element within the interactive scene at a predetermined position and an orientation; and
providing, based on the predetermined position, a second indication that the capturing will likely fail.

8. The device of claim 1 , wherein providing the first indication comprises:
determining a perceived distance to the augmented reality element relative to a viewpoint for the interactive scene.

9. A method comprising:
displaying on a display screen an interactive scene containing an augmented reality element and a region represented by a first graphical user interface element;
receiving an input from a user, the input generated from a motion associated with orienting and positioning the display screen to place the augmented reality element at least partially within the region;
providing a first indication of whether a capturing of an image of an object in a position of the augmented reality element may enable successful recognition of the object by a recognition model, the first indication based on determining a likelihood of successful recognition of the object by:
determining a match score based on a metric associated with the placement of the augmented reality element at least partially within the region; and
determining whether the match score is above a threshold value; and

displaying suggestions for manipulating the device based on the match score, the suggestions including virtually rotating the augmented reality element to simulate a repositioning of the device.

10. The method of claim 9 , wherein the input comprises at least one of:
zooming a viewpoint to the augmented reality element;
rotating the viewpoint for the interactive scene;
moving the viewpoint for the interactive scene; or
changing lighting conditions of the interactive scene.

11. The method of claim 9 , wherein displaying the interactive scene comprises displaying a view of an environment of the user.

12. The method of claim 10 , comprising:
performing the method on a device comprising a smartphone; and
moving the smartphone so as to rotate the viewpoint.

13. The method of claim 12 , wherein moving the smartphone comprises rotating the smartphone around an axis of the smartphone.

14. The method of claim 12 , further comprising providing a second indication of how to position the device for a successful capturing of an image of the object.

15. The method of claim 14 , wherein the second indication comprises an arrow indicating where to move the device.

16. The method of claim 9 , wherein:
the region comprises at least one of:
a transparent region having a boundary;
a semi-transparent region; or
a set of disconnected boundary elements and the region has a rectangular shape.

17. A method comprising:
displaying on a display screen an interactive scene containing an augmented reality element;
receiving a command from a user for simulating capturing of image data, the command comprising an interaction of the user with the display screen via a graphical user interface;
calculating a match score representing a likelihood that an object in the place of the augmented reality element would be recognized by an object recognition model;
comparing the match score with a predefined threshold; and
suggesting to the user a device manipulation for improving the match score, based on a value of the match score, the suggesting including virtually rotating the augmented reality element to simulate a repositioning of the device.

18. The method of claim 17 , wherein the device manipulation comprises at least one of:
moving away from the augmented reality element;
approaching the augmented reality element; or
moving laterally relative to the augmented reality element.

19. The method of claim 17 , further comprising providing an indication of whether the simulating capturing of the image data is successful.

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Blacksher Victoria
• Castaneda Jonathan
• Youngblood Stewart

Classifications

• G06V10/235

  based on user input or interaction · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title