Systems and methods for facilitating a transaction relating to newly identified items using augmented reality

Patent metadata

Field	Value
Publication number	US-10007948-B1
Application number	US-201715667043-A
Country	US
Kind code	B1
Filing date	Aug 2, 2017
Priority date	Dec 22, 2016
Publication date	Jun 26, 2018
Grant date	Jun 26, 2018

Abstract

A system may provide an augmented environment that facilitates a transaction. The system may store profile data including user payment or user profile information. The system may then receive environmental data including audio and visual information representing a physical environment. The system may then receive first user input data indicative of a selection of one or more items present in the physical environment, and identify one or more action items in the environmental data based on the first user input data. In response to this identification, the system may augment the environmental data by adding virtual environmental data, and then provide this virtual environmental data to a device to create an augmented environment. The system can then receive second user input data, and provide purchase request data to a merchant terminal to enable a transaction related to the one or more action items.

First claim

The invention claimed is:

1. A system for facilitating a transaction, comprising:
a memory for storing profile data comprising user payment credentials and user profile information;
an environmental data sensor for obtaining environmental data comprising audio and/or visual information representing a physical environment;
a user interface device for generating user input data in response to one or more user inputs, including first user input data indicative of a selection of one or more items present in the physical environment, the first user input data being derived from an image of the one or more items present in the physical environment obtained by the environmental data sensor based on a first gesture made in association with the one or more action items;
an augmented environment processor that, based on the first user input data and environmental data obtained from the environmental data sensor, identifies one or more action items in the environmental data that are associated with the one or more selected items based on an identification of the more or more selected items using image recognition techniques on the image, augments the environmental data by adding the virtual environmental data comprising enhanced content associated with one or more of the one or more action items, generates a signal comprising augmented environmental data, and generates purchase request data based on the one or more action items, the user profile information, and second user input data associated with one or more action items generated by the user interface device;
an augmented environmental data display that receives the signal comprising augmented environmental data and displays said augmented environmental data in accordance with said signal; and
an input/output device for transmitting the purchase request data to a merchant terminal;
wherein the enhanced content is unrepresented in the physical environment.

2. The system of claim 1 , wherein the second user input data is generated in response to a second gesture indicating an instruction to purchase an item.

3. A system for facilitating a transaction, comprising:
one or more processors; and
a memory in communication with the one or more processors and storing instructions that, when executed by the one or more processors, are configured to cause the system to:
store profile data comprising user payment credentials and user profile information;
receive, from a computing device associated with the stored profile data, environmental data comprising audio and visual information representing a physical environment;
receive, from the computing device, first user input data indicative of a selection of one or more items present in the physical environment, the first user input data being derived from an image of the one or more items present in the physical environment obtained by a camera associated with the computing device based on a gesture made in association with the one or more items that is detected by the computing device;
identify one or more action items in the environmental data based on the first user input data, the one or more action items being associated with the one or more selected items, the identification of the one or more action items being based on an identification of the one or more selected items using image recognition techniques on the image;
augment the environmental data by adding virtual environmental data comprising enhanced content associated with one or more of the one or more action items, wherein the enhanced content is unrepresented in the physical environment;
provide, to the computing device, the virtual environmental data to create an augmented environment in combination with the physical environment;
receive second user input data associated with the one or more action items from the computing device; and
provide, based on the second user input data, the one or more actions items, and the user profile information, purchase request data to a merchant terminal to enable a transaction related to the one or more action items, wherein the purchase request data comprises the stored user payment credentials.

4. The system of claim 3 , wherein the transaction related to the one or more action items represents a purchase of the one or more selected items on behalf of a user of the computing device.

5. The system of claim 3 , wherein the transaction related to the one or more action items represents a purchase of the one or more selected items on behalf of someone other than a user of the computing device.

6. The system of claim 3 , wherein the one or more processors are further configured to execute the instructions to cause the system to receive, from a merchant terminal, one or more 3-dimensional models of the one or more selected items to aid in performing image recognition.

7. The system of claim 3 , wherein identifying one or more action items in the environmental data based on the first user input data comprises detecting beacons.

8. The system of claim 3 , wherein the first user input data is further derived from an audible command detected by the computing device.

9. The system of claim 3 , wherein the gesture comprises focusing on an item for greater than a predetermined threshold period of time.

10. The system of claim 3 , wherein the gesture comprises making a grabbing motion towards an item.

11. The system of claim 3 wherein the second user input data is derived from an authenticating gesture.

12. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a computer system to:
store profile data comprising user payment credentials and user profile information;
receive, from a computing device associated with the stored profile data, environmental data comprising audio and visual information representing a physical environment;
receive, from the computing device, first user input data indicative of a selection of one or more items present in the physical environment, the first input data being derived from an image of the one or more items present in the physical environment obtained by a camera associated with the computing device based on a gesture made in association with the one or more items that is detected by the computing device;
identify one or more action items in the environmental data based on the first user input data, the one or more action items being associated with the one or more selected items, the identification of the one or more action items being based on an identification of the one or more selected items using image recognition techniques on the image;
augment the environmental data by adding virtual environmental data comprising enhanced content associated with the one or more action items, wherein the enhanced content is unrepresented in the physical environment;
provide, to the computing device, the virtual environmental data to create an augmented environment in combination with the physical environment;
receive second user input data associated with the one or more action items from the computing device; and
provide, based on the second user input data, the one or more actions items, and the user profile information, purchase request data to a merchant terminal to enable a transaction related to the one or more action items, wherein the purchase request data comprises the stored user payment credentials.

13. The non-transitory computer-readable medium of claim 12 , wherein the transaction related to the one or more action items represents a purchase of the one or more selected items on behalf of a user of the computing device.

14. The non-transitory co

Assignees

• Capital One Services Llc

Inventors

• Nickerson Karen
• Wishne Justin
• Jacobs Drew
• Smith Justin
• Giampaolo Marco S
• Jouhikainen Hannes

Classifications

• G06Q20/12

  specially adapted for electronic shopping systems · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06Q30/0641

  utilising user interfaces specially adapted for shopping · CPC title

• G06Q20/202

  Interconnection or interaction of plural electronic cash registers [ECR] or to host computer, e.g. network details, transfer of information from host to ECR or from ECR to ECR · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title