Methods and systems for providing an augmented reality interface for saving information for recognized objects

Patent metadata

Field	Value
Publication number	US-11204974-B2
Application number	US-201916516453-A
Country	US
Kind code	B2
Filing date	Jul 19, 2019
Priority date	Jul 19, 2019
Publication date	Dec 21, 2021
Grant date	Dec 21, 2021

Abstract

A method includes displaying and capturing image data containing an object and accessing a plurality of records related to objects, selecting a record related to the captured object, obtaining an identifier of a vendor of the object of the selected data record, combining the selected data record and the vendor identifier to form a search record, displaying, based on the search record, an augmented reality interface to receive a first interactive action for saving the search record, receiving the first interactive action, saving, in response to receiving the first interactive action, the search record into a searchable data structure, receiving a second interactive action, retrieving, in response to receiving the second interactive action, the search record from the searchable data structure, updating the vendor identifier based on the retrieved search record, and displaying information related to the search record.

First claim

What is claimed is:

1. A method for processing information about an object requiring recognition, comprising:
displaying a scene containing the object;
receiving a request to capture image data of the scene at a first time and a first location;
capturing the requested image data;
extracting image data of the object from the captured image data;
accessing a plurality of data records, the data records comprising image data of stored objects;
identifying probabilities that the data records are related to the image data of the object based on one or more features identified within the image data;
selecting one of the data records based on the highest probability;
obtaining a vendor identifier of a first vendor of the stored object of the selected data record;
combining the selected data record and the vendor identifier to form a search record;
displaying, based on the search record and only when a probability that a data record is related to image data of the object is at least equal to a predetermined threshold, an augmented reality interface to receive a first interactive action for saving the search record;
receiving the first interactive action;
saving, in response to receiving the first interactive action, the search record into a searchable data structure;
receiving a second interactive action to perform an updated vendor search based on the search record at a second time and a second location;
retrieving, in response to receiving the second interactive action, the search record from the searchable data structure;
updating, when at least one of the second time or the second location is sufficiently different from the corresponding one of the first time or the first location, the vendor identifier of the retrieved search record, wherein updating the vendor identifier includes obtaining an identifier for a second vendor based on the second location at which the second interactive action is performed; and
displaying information related to the updated vendor search.

2. The method of claim 1 , wherein obtaining the identifier for the first or the second vendor comprises obtaining a name and a location of the first or second vendor, respectively.

3. The method of claim 2 , wherein the obtaining the name and the location of the first or the second vendor comprises:
selecting a chosen location;
selecting a chosen radial distance about the chosen location; and
selecting the vendor of the stored object of the selected data record, the vendor having a location at a distance from the chosen location that is no larger than the chosen radial distance.

4. The method of claim 1 , further comprising obtaining a plurality of identifiers for a plurality of vendors.

5. The method of claim 1 , further comprising obtaining a price of the stored object requested by the first or second vendor.

6. The method of claim 5 , wherein the augmented reality interface comprises an interface for making a comment, the comment being at least one of text data or graphics data related to at least one of the stored objects, a vendor, or the price of the stored object.

7. The method of claim 1 , wherein the augmented reality interface comprises an interface for editing the search record.

8. The method of claim 1 , wherein updating the identifier comprises evaluating the probabilities using an artificial neural network.

9. The method of claim 1 , wherein displaying the augmented reality interface comprises
providing an indication when the probability is less than the predetermined threshold.

10. The method of claim 1 , further comprising using a computer-based neural network model to perform the step of selecting the one of the data records.

11. The method of claim 1 , wherein at least one of the first or the second interactive action comprises a gesture from a user.

12. The method of claim 1 , wherein the augmented reality interface comprises a graphical overlay.

13. The method of claim 1 , wherein capturing the requested image data comprises manipulating a viewpoint of the scene, wherein manipulating the viewpoint of the scene includes orienting a camera for capturing one or more unique features for identifying the object.

14. A method for processing information about an object requiring recognition, comprising:
displaying a scene containing the object;
receiving a request to capture image data of the scene at a first time and a first location;
capturing the requested image data;
extracting image data of the object from the captured image data;
accessing a plurality of data records, the data records comprising image data of stored objects;
identifying probabilities that the data records are related to the image data of the object based on one or more features identified within the image data;
selecting one of the data records based on the highest probability;
obtaining a vendor identifier of a first vendor of the stored object of the selected data record;
combining the selected data record and the vendor identifier to form a search record accessible by a computing device;
displaying, based on the search record and only when a probability that a data record is related to image data of the object is at least equal to a predetermined threshold, an augmented reality interface to receive a first interactive action for saving the search record;
receiving the first interactive action to perform an updated vendor search based on the search record;
saving the search record into a database, in response to receiving the first interactive action;
receiving a request at a second time and a second location to retrieve the search record using the computing device;
retrieving the search record from the database, in response to the received request;
updating, when at least one of the second time or the second location is sufficiently different from the corresponding one of the first time or the first location, the vendor identifier of the retrieved search record, wherein updating the vendor identifier includes obtaining an identifier for a second vendor based on the second location at which the second interactive action is performed; and
displaying information related to the search record.

15. The method of claim 14 , wherein identifying probabilities that the data records are related to the stored object comprises:
transmitting the extracted image data of the object to a remote computing system; and
receiving the identified data record having a highest probability from the remote computing system.

16. A non-transitory computer-readable medium containing instructions, which when executed by a processor result in operations comprising:
displaying a scene containing an object;
receiving a request to capture image data of the scene at a first time and a first location;
capturing the requested image data;
extracting image data of the object from the captured image data;
accessing a plurality of data records, the data records comprising image data of stored objects;
identifying probabilities that the data records are related to the image data of the object based on one or more features identified within the image data;
selecting one of the data records based on the highest probability;
obtaining a vendor identifier of a first vendor of the stored object of the selected data record;
combining the selected data record and the vendor identifier to form a search record;
displaying, based on the search record and only when a probability that a data record is related to image data of a stored object is at least equal to a predetermined threshold, an augmented reality interface to receive a fir

Assignees

• Capital One Services Llc

Inventors

• Duckworth Staevan
• Martinez Daniel
• Hardin William
• Blacksher Victoria
• Castaneda Jonathan
• Youngblood Stewart

Classifications

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06V2201/08

  Detecting or categorising vehicles · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06Q30/0625 Primary

  by formulating product or service queries, e.g. using keywords or predefined options · CPC title

• G06F16/532

  Query formulation, e.g. graphical querying · CPC title