Methods and systems for providing a tutorial for graphic manipulation of objects including real-time scanning in an augmented reality

Patent metadata

Field	Value
Publication number	US-2020410768-A1
Application number	US-202016911803-A
Country	US
Kind code	A1
Filing date	Jun 25, 2020
Priority date	Jun 28, 2019
Publication date	Dec 31, 2020
Grant date	—

Abstract

A device including a storage medium storing instructions, and at least one processor configured to execute the stored instructions to perform operations is provided. The operations include displaying on a display screen an interactive scene containing an augmented reality element, receiving input to change display of the interactive scene, and receiving a request to simulate the capturing of image data representing the interactive scene. The operations further include determining a position of the augmented reality element relative to a region defined on the display screen, simulating the capturing of the image data representing the interactive scene, and providing, based on the determined position, an indication of whether the simulation has succeeded.

First claim

1 - 20 . (canceled)

21 . A device, comprising:
a storage medium storing instructions; and at least one processor configured to execute the stored instructions to perform operations comprising:
displaying on a display screen an interactive scene containing an augmented reality element;
receiving an input from a user, the input generated from a motion associated with orienting and positioning the display screen to place the augmented reality element at least partially within a region represented by a first graphical user interface element shown on the display screen; and
providing an indication of whether a capturing of an object in a position of the augmented reality element will enable successful recognition of the object, wherein the successful recognition of the object is determined by:
identifying a probability of determining an identity of the object based on a location of the object within the region; and
determining whether the probability is above a threshold value.

22 . The device of claim 21 , wherein the input comprises at least one of zooming a viewpoint to the augmented reality element, rotating the viewpoint for the interactive scene, moving the viewpoint for the interactive scene, or changing lighting conditions of the interactive scene.

23 . The device of claim 21 , wherein the operations further comprise:
receiving a request to retrieve the augmented reality element from a database; and retrieving the augmented reality element for the database.

24 . The device of claim 21 , wherein displaying the interactive scene comprises displaying a view of an environment of the user.

25 . The device of claim 21 , wherein providing the indication of whether the capturing will enable successful recognition of the object comprises:
identifying a data record in a database corresponding to the object.

26 . The device of claim 21 , wherein the augmented reality element corresponds to a virtual vehicle.

27 . The device of claim 21 , wherein the operations further comprise:
initially displaying the augmented reality element within the interactive scene at a predetermined position and an orientation; and providing, based on the predetermined position, an indication that the capturing will likely fail.

28 . The device of claim 21 , wherein the operations further comprise:
displaying suggestions for manipulating the device based on the indication of whether the capturing will enable successful recognition of the object.

29 . The device of claim 21 , wherein providing the indication of whether the capturing will enable successful recognition of the object comprises:
determining a perceived distance to the augmented reality element relative to a viewpoint for the interactive scene.

30 . A method comprising:
displaying on a display screen an interactive scene containing an augmented reality element; receiving an input from a user, the input generated from a motion associated with orienting and positioning the display screen to place the augmented reality element at least partially within a region represented by a first graphical user interface element shown on the display screen; and providing an indication of whether a capturing of an object in a position of the augmented reality element will enable successful recognition of the object, wherein the successful recognition of the object is determined by: identifying a probability of determining an identity of the object; and determining whether the probability is above a threshold value.

31 . The method of claim 30 , wherein the input comprises at least one of zooming a viewpoint to the augmented reality element, or rotating the viewpoint for the interactive scene, moving the viewpoint for the interactive scene, or changing lighting conditions of the interactive scene.

32 . The method of claim 30 , wherein displaying on the display screen the interactive scene comprises displaying a view of an environment of the user.

33 . The method of claim 31 , wherein a device used for the method comprises a smartphone, and wherein rotating the viewpoint for the interactive scene includes moving the smartphone.

34 . The method of claim 33 , wherein moving the smartphone comprises rotating the smartphone around one of smartphone axes.

35 . The method of claim 33 , further comprising providing an indication of how to position the device for a successful capturing of the object.

36 . The method of claim 35 , wherein the indication comprises an arrow indicating where to move the device.

37 . The method of claim 30 , wherein the region is one of a transparent region having a boundary, a semi-transparent region, or a set of disconnected boundary elements, and wherein the region has a rectangular shape.

38 . A method comprising:
displaying on a display screen an interactive scene containing an augmented reality element; receiving a command from a user for simulating capturing of image data, the command comprising the user interacting with the display screen via a graphical user interface; calculating a match score describing a likelihood that an object in the place of the augmented reality element would be recognized by an object recognition model; comparing the match score with a predefined threshold; and suggesting to the user a device manipulation for improving the match score, based on a value of the match score.

39 . The method of claim 38 , wherein the device manipulation comprises one of a moving away from the augmented reality element, approaching the augmented reality element, or moving laterally relative to the augmented reality element.

40 . The method of claim 38 , further comprising providing an indication of whether the simulating capturing of the image data is successful.

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Blacksher Victoria
• Castaneda Jonathan
• Youngblood Stewart

Classifications

• G06V10/235

  based on user input or interaction · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06T15/50

  Lighting effects · CPC title