Object recognition and analysis using augmented reality user devices

Patent metadata

Field	Value
Publication number	US-10679272-B2
Application number	US-201916374851-A
Country	US
Kind code	B2
Filing date	Apr 4, 2019
Priority date	Nov 30, 2016
Publication date	Jun 9, 2020
Grant date	Jun 9, 2020

Abstract

An augmented reality system that includes an augmented reality user device. The augmented reality user device includes a display for overlaying virtual objects onto objects in a real scene and a camera. The augmented reality user device includes a processor implementing a virtual assessment engine and a virtual overlay engine. The virtual assessment engine authenticates the user and identifies a user identifier for the user. The virtual assessment engine captures an image and performs object recognition on the image to identify products in a cart. The virtual assessment engine sends a token to a remote server, receives new account offers for the user from the server, and presents the offers as virtual objects overlaid with the products in the cart.

First claim

What is claimed is:

1. An augmented reality system comprising:
an augmented reality user device for a user comprising:
a display configured to overlay virtual objects onto tangible objects in a real scene in real-time;
a camera configured to capture images of tangible products;
one or more processors operably coupled to the display and the camera, and configured to implement:
a virtual assessment engine configured to:
authenticate the user based on a user input;
identify a user identifier for the user in response to authenticating the user;
capture an image of a product;
perform object recognition on the image to identify the product;
determine a price of the identified product;
generate a token comprising:
 the user identifier, and
 the price of the identified product;
send the token to a remote server;
receive virtual assessment data in response to sending the token, wherein the virtual assessment data comprises a recommendation identifying a selected account for the user and one or more new prequalified accounts for the user; and

a virtual overlay engine configured to present the recommendation identifying the selected account and the one or more new prequalified accounts as virtual objects overlaid with the product; and

the remote server comprising a product analysis engine configured to:
receive the token;
identify account information comprising one or more existing accounts for the user based on the user identifier;
prequalify the user for one or more new accounts based on the account information;
select an account from the one or more existing accounts and the one or more prequalified new accounts from the one or more new accounts for the user based on the price of the identified product;
generate the recommendation that identifies the selected account and the selected one or more prequalified new accounts;
generate the virtual assessment data identifying the recommendation; and
send the virtual assessment data to the augmented reality user device.

2. The system of claim 1 , wherein:
the token comprises a product identifier for the identified product;
the product analysis engine is configured to:
identify one or more alternative vendors based on the product identifier; and
identify alternative prices for each of the alternative vendors; and

the virtual assessment data identifies the alternative vendors and the alternative prices.

3. The system of claim 1 , wherein:
the token comprises a product identifier for the identified product;
the product analysis engine is configured to:
identify an alternative vendor based on the product identifier; and
obtain an alternative price for the product from a third-party database linked with the alternative vendor; and

the virtual assessment data identifies the alternative vendor and the alternative price.

4. The system of claim 1 , wherein the virtual assessment data comprises historical pricing information for the identified product.

5. The system of claim 1 , wherein:
the token comprises a product identifier for the identified product;
the product analysis engine is configured to identify product information for the product based on the product identifier; and
the virtual assessment data comprises the product information for the identified product.

6. The system of claim 1 , wherein:
the augmented reality user device comprises a voice recognition engine configured to identify a voice command performed by the user indicating a selected new account from the one or more available new accounts; and
the virtual assessment engine is configured to send an activation command identifying the selected new account to the remote server.

7. The system of claim 1 , wherein:
the augmented reality user device comprises a gesture recognition engine configured to identify a gesture performed by the user indicating a selected new account from the one or more available new accounts; and
the virtual assessment engine is configured to send an activation command identifying the selected new account to the remote server.

8. An augmented reality overlaying method comprising:
authenticating, by a virtual assessment engine of an augmented reality user device, a user based on a user input;
identifying, by the virtual assessment engine, a user identifier for the user in response to authenticating the user;
capturing, by a camera operably coupled to the virtual assessment engine, an image of a product;
performing, by the virtual assessment engine, object recognition on the image to identify the product;
determining, by the virtual assessment engine, a price of the identified product;
generating, by the virtual assessment engine, a token comprising:
the user identifier, and
the price of the identified product;

sending, by the virtual assessment engine, the token to a remote server;
receiving, by a product analysis engine of the remote server, the token;
identifying, by the product analysis engine of the remote server, account information comprising one or more existing accounts for the user based on the user identifier;
prequalifying, by the product analysis engine, the user for one or more new accounts based on the account information;
selecting, by the product analysis engine, an account from the one or more existing accounts and one or more prequalified new accounts from the one or more new accounts for the user based on the price of the identified product;
generating, by the product analysis engine, a recommendation that identifies the selected account and the selected one or more prequalified new accounts;
generating, by the product analysis engine, virtual assessment data comprising the recommendation;
sending, by the product analysis engine, the virtual assessment data to the augmented reality user device; and
presenting, by a virtual overlay engine of the augmented reality user device, the recommendation identifying the selected account and the one or more new prequalified accounts as virtual objects overlaid with the product.

9. The method of claim 8 , wherein:
the token comprises a product identifier for the identified product; and
the virtual assessment data identifies one or more alternative vendors and alternative prices for each of the alternative vendors; and
further comprising:
identifying, by the product analysis engine, the one or more alternative vendors based on the product identifier; and
identifying, by the product analysis engine, the alternative prices for each of the alternative vendors.

10. The method of claim 8 , wherein:
the token comprises a product identifier for the identified product; and
the virtual assessment data identifies an alternative vendor and an alternative price for the alternative vendors; and
further comprising:
identifying, by the product analysis engine, the alternative vendor based on the product identifier; and
obtaining, by the product analysis engine, the alternative price for the identified product from a third-party database linked with the alternative vendor.

11. The method of claim 8 , wherein the virtual assessment data comprises historical pricing information for the identified product.

12. The method of claim 8 , wherein:
the token comprises a product identifier for the identified product;
the virtual assessment data comprises product information for the identified product; and
further comprising identifying, by the product analysis engine, the product information for the identified product based on the product identifier.

13. The method of claim 8 , further comprising:
identifying, by a voice recognition engine, a voice command performed by the user indicat

Assignees

• Bank Of America

Inventors

• Waldron Willard H
• Dravneek Victoria L
• Dintenfass Katherine
• Wyllie Graham M
• Kaye Kenneth A
• Dolan Heather

Classifications

• H04L63/08

  for authentication of entities (cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32) · CPC title

• G06Q30/0623 Primary

  by investigating goods or services · CPC title

• H04L67/18

  Electricity · mapped topic

• G06Q40/02

  Banking, e.g. interest calculation or account maintenance (credit or loans G06Q40/03) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title