Virtual assessments using augmented reality user devices

Patent metadata

Field	Value
Publication number	US-10685386-B2
Application number	US-201615364927-A
Country	US
Kind code	B2
Filing date	Nov 30, 2016
Priority date	Nov 30, 2016
Publication date	Jun 16, 2020
Grant date	Jun 16, 2020

Abstract

An augmented reality system that includes an augmented reality user device. The augmented reality user device includes a display for overlaying virtual objects onto objects in a real scene, a camera, and a global position system sensor. The augmented reality user device includes a processor implementing an object recognition engine, a virtual assessment engine, and a virtual overlay engine. The object recognition engine identifies tangible products from an image. The virtual assessment engine authenticates the user, identifies a user identifier for the user, and identifies a vendor based on the geographic location of the user. The virtual assessment engine captures an image and performs object recognition on the image to identify products in a cart. The virtual assessment engine sends a token to a remote server, receives available offers for the user from the server, and presents the available offers as virtual objects overlaid with the products in the cart.

First claim

The invention claimed is:

1. An augmented reality system comprising:
an augmented reality user device for a user, wherein the augmented reality user device is a mobile user device associated with the user comprising:
a network interface;
a display configured to overlay virtual objects onto tangible objects in a real scene in real-time;
a camera configured to capture images of tangible products;
a global position system (GPS) sensor configured to provide a geographic location of the user;
one or more processors operably coupled to the display, the camera, and the GPS sensor, and configured to:
authenticate the user based on a user input;
identify a user identifier for the user in response to authenticating the user;
determine the geographic location of the user using the GPS sensor;
identify a vendor based on the geographic location of the user;
capture an image of products in a cart using the camera;
perform object recognition on the image to identify the products in the cart;
generate a token comprising:
the user identifier,
a vendor identifier identifying the vendor, and
product identifiers identifying the products;

send the token to a remote server using the network interface;
receive in response to sending the token, virtual assessment data comprising:
alternative vendors for one or more of the products;
alternative prices for the one or more products corresponding with the alternative vendors;
one or more available offers for the user; and
account information for the user; and

present the alternative vendors, the alternative prices, the one or more available offers, and the account information for the user as virtual objects overlaid with the products in the cart using the display; and

the remote server comprising one or more processors:
receive the token;
determine account information for the user based on the user identifier;
identify the one or more available offers for the user based on at least one of the vendor identifier, the product identifiers, and the account information for the user;
identify alternative vendors for the products based on the product identifiers;
identify alternative prices for the products corresponding with the alternative vendors;
generate the virtual assessment data; and
send the virtual assessment data to the augmented reality user device.

2. The system of claim 1 , wherein:
the user input is a voice command; and
the one or more processors of the augmented reality user device are configured to perform voice recognition on the voice command to authenticate the user.

3. The system of claim 1 , wherein:
the user input is a biometric signal; and
the one or more processors of the augmented reality user device are configured to use the biometric signal to authenticate the user.

4. The system of claim 1 , wherein identifying the one or more available offers for the user comprises receiving at least one offer from a third-party database.

5. The system of claim 1 , wherein:
the virtual assessment data comprises budget information for the user; and
the one or more processors of the augmented reality device are configured to present the budget information for the user.

6. The system of claim 1 , wherein the one or more processors of the augmented reality user device are configured to:
identify a voice command performed by the user indicating a selected offer from the one or more available offers; and
send an activation command identifying the selected offer to the remote server.

7. The system of claim 1 , wherein the one or more processors of the augmented reality user device are configured to:
identify a gesture performed by the user indicating a selected offer from the one or more available offers; and
send an activation command identifying the selected offer to the remote server.

8. An augmented reality overlaying method comprising:
authenticating, by an augmented reality user device, a user based on a user input, wherein the augmented reality user device is a mobile device associated with the user;
identifying, by the augmented reality user device, a user identifier for the user in response to authenticating the user;
determining, by the augmented reality user device, a geographic location of the user using a global position system (GPS) sensor;
identifying, by the augmented reality user device, a vendor based on the geographic location of the user;
capturing, by the augmented reality user device, an image of products in a cart using a camera;
performing, by the augmented reality user device, object recognition on the image to identify the products in the cart;
generating, by the augmented reality user device, a token comprising the user identifier, a vendor identifier identifying the vendor, and product identifiers identifying the products;
sending, by the augmented reality user device, the token to a remote server using a network interface;
determining, by the remote server, account information for the user based on the user identifier;
identifying, by the remote server, one or more available offers for the user based on at least one of the vendor identifier, the product identifiers, and the account information for the user;
identifying, by the remote server, alternative vendors for the products based on the product identifiers;
identifying, by the remote server, alternative prices for the products corresponding with the alternative vendors;
generating, by the remote server, the virtual assessment data comprising the alternative vendors, the alternative prices, the one or more available offers for the user, and account information for the user in response to sending the token;
sending, by the remote server, the virtual assessment data to the augmented reality user device;
presenting, by the augmented reality user device, the alternative vendors, the alternative prices, the one or more available offers, and the account information for the user as virtual objects overlaid with the products in the cart using a display.

9. The method of claim 8 , wherein the user input is a voice command; and
further comprising performing, by the augmented reality user device, voice recognition on the voice command to authenticate the user.

10. The method of claim 8 , wherein the user input is a biometric signal; and
further comprising using, by the augmented reality user device, the biometric signal to authenticate the user.

11. The method of claim 8 , wherein identifying the one or more available offers for the user comprises receiving at least one offer from a third-party database.

12. The method of claim 8 , further comprising:
identifying, by the augmented reality user device, a voice command performed by the user indicating a selected offer from the one or more available offers; and
sending, by the augmented reality user device, an activation command identifying the selected offer to the remote server.

13. The method of claim 8 , further comprising:
identifying, by the augmented reality user device, a gesture performed by the user indicating a selected offer from the one or more available offers; and
sending, by the augmented reality user device, an activation command identifying the selected offer to the remote server.

14. An augmented reality device for a user comprising:
a network interface;
a display configured to overlay virtual objects onto tangible objects in a real scene in real-time;
a camera configured to capture images of tangible products;
a global position system (GPS) sensor configured to provide a geographic location of the user;
one or more processors operably coupled to the display, the ca

Assignees

• Bank Of America

Inventors

• Waldron Willard H
• Dravneek Victoria L
• Dintenfass Katherine
• Wyllie Graham M
• Kaye Kenneth A
• Dolan Heather

Classifications

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06Q30/0633 Primary

  Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087) · CPC title

• G06Q30/0207

  Discounts or incentives, e.g. coupons or rebates · CPC title

• H04L63/0861

  using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231) · CPC title