Systems and methods of sharing an augmented environment with a companion

Patent metadata

Field	Value
Publication number	US-11074647-B2
Application number	US-201916445333-A
Country	US
Kind code	B2
Filing date	Jun 19, 2019
Priority date	Dec 22, 2016
Publication date	Jul 27, 2021
Grant date	Jul 27, 2021

Abstract

A system includes one or more memory devices storing instructions, and one or more processors configured to execute the instructions to perform steps of a method for providing an interactive virtual environment. The system may receive environmental data, and augment the environmental data by adding virtual environmental data. The system may then provide this virtual environment to two or more devices. The system can then receive user input, and modify the virtual environment in response to those inputs.

First claim

What is claimed is:

1. A system for providing an interactive augmented environment, the system comprising:
one or more processors; and
memory comprising instructions that, when executed by the one or more processors, cause the system to:
receive, from a first computing device, first environmental data comprising first visual information representing a physical environment;
identify, based on the first environmental data, a perceived environmental indicator indicative of a virtual location corresponding to a physical location in the physical environment;
augment the first environmental data by adding, based on the perceived environmental indicator, virtual environmental data associated with the perceived environmental indicator and comprising indicator display data corresponding to the perceived environmental indicator, the indicator display data being unrepresented in the physical environment;
transmit, to the first computing device, the virtual environmental data to create an augmented environment that combines the virtual environmental data with the first environmental data;
receive, from a second computing device, second environmental data comprising second visual information representing the physical environment;
identify, based on the second environmental data, the perceived environmental indicator;
transmit, to the second computing device, the virtual environmental data associated with the perceived environmental indicator to create an augmented environment combining the virtual environmental data with the second environmental data;
receive, from the first computing device or the second computing device, user input data comprising an instruction to modify the indicator display data;
modify, based on the user input data, the indicator display data to form modified virtual environmental data; and
transmit, to the first computing device, the modified virtual environmental data to update the augmented environment to display the modified indicator display data,
wherein identifying the perceived environmental indicator comprises receiving, from the first computing device, a request to place a virtual marker at the virtual location corresponding to the physical location in the physical environment.

2. The system of claim 1 , wherein the perceived environmental indicator corresponds to a physical indicator present in the physical environment.

3. The system of claim 2 , wherein the physical indicator is associated with a physical product for purchase.

4. The system of claim 1 , wherein the physical environment is a merchant location associated with a merchant.

5. The system of claim 1 , wherein the virtual environmental data includes one or more of product price information, product inventory information, product manufacturing information, product marketing information or description, product ratings or reviews, social information, related products, available discounts, and aesthetic visual effects.

6. The system of claim 1 , wherein the first and second computing devices each comprise a mobile phone, a virtual reality headset, an augmented reality headset, a wearable device, an internet of things (IoT) device, a tablet computer, a personal computer, or combinations thereof.

7. The system of claim 1 , wherein augmenting the first environmental data further comprises identifying one or more physical objects in the first visual information.

8. The system of claim 7 , wherein augmenting the first environmental data further comprises displaying information from a merchant that corresponds to at least one of the one or more physical objects identified in the first visual information.

9. The system of claim 1 , wherein the user input data is associated with one or more predefined user inputs.

10. The system of claim 9 , wherein the one or more predefined user inputs comprises a selection of a product displayed in the augmented environment.

11. The system of claim 9 , wherein the one or more predefined user inputs comprises a modification of a product displayed in the augmented environment.

12. The system of claim 11 , wherein the modification of a product displayed in the virtual environment comprises one or more of a color change, a size change, and a style change.

13. The system of claim 11 , wherein the instructions, when executed by the one or more processors, further cause the system to:
receive, from the first computing device or the second computing device, a purchase request to purchase a product associated with the perceived environmental indicator;
transmit, to a financial service provider, a purchase authorization request corresponding to the purchase request to purchase the product associated with the perceived environmental indicator; and
responsive to receiving authorization from the financial service provider, transmit the purchase request to a merchant.

14. A method, performed by one or more processors, for providing an interactive augmented environment, the method comprising:
receiving, from a first computing device, first environmental data comprising first visual information representing a physical environment;
identifying, based on the first environmental data, a perceived environmental indicator indicative of a virtual location corresponding to a physical location in the physical environment;
augmenting the first environmental data by adding, based on the perceived environmental indicator, virtual environmental data associated with the perceived environmental indicator and comprising data associated with a merchant and indicator display data corresponding to the perceived environmental indicator, the indicator display data being unrepresented in the physical environment;
transmitting, to the first computing device, the virtual environmental data to create an augmented environment that combines the virtual environmental data with the first environmental data;
receiving, from a second computing device, second environmental data comprising second visual information representing the physical environment;
identifying, based on the second environmental data, the perceived environmental indicator;
transmitting, to the second computing device, the virtual environmental data associated with the perceived environmental indicator to create an augmented environment combining the virtual environmental data with the second environmental data;
receiving, from the first computing device or the second computing device, user input data comprising an instruction to modify the indicator display data;
modifying, based on the user input data, the indicator display data to form modified virtual environmental data; and
transmitting, to the first computing device, the modified virtual environmental data to update the augmented environment to display the modified indicator display data,
wherein identifying the perceived environmental indicator comprises receiving, from the first computing device, a request to place a virtual marker at the virtual location corresponding to the physical location in the physical environment.

15. The method of claim 14 , wherein the perceived environmental indicator corresponds to a physical indicator present in the physical environment.

16. The method of claim 15 , wherein the physical indicator is associated with a physical product for purchase.

17. The method of claim 14 , wherein the first and second computing devices each comprise a mobile phone, a virtual reality headset, an augmented reality headset, a wearable device, an internet of things (IoT) device, a tablet computer, a personal computer, or combinations thereof.

Assignees

• Capital One Services Llc

Inventors

• Nickerson Karen
• Wishne Justin
• Jacobs Drew
• Smith Justin
• Giampaolo Marco S
• Jouhikainen Hannes

Classifications

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G09G2370/02

  Networking aspects · CPC title

• G06F3/147

  using display panels · CPC title

• G06Q30/0643 Primary

  graphically representing goods, e.g. 3D product representation · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title