Method and system for generating a retail experience using an augmented reality system

Patent metadata

Field	Value
Publication number	US-11221213-B2
Application number	US-201514707513-A
Country	US
Kind code	B2
Filing date	May 8, 2015
Priority date	Jul 12, 2013
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

An augmented reality AR system and method for a retail experience include a waveguide apparatus that includes a planar waveguide and at least one optical diffraction element The AR retail system and method recognizes user location in a retail establishment, retrieves data corresponding to the retail establishment and generates virtual content relating to the retail establishment based on the retrieved data. The AR retail system and method creates a virtual user interface in a user's field of view. Virtual content is displayed on the virtual user interface while the user is engaged in retail activity and may be based on user input. The AR retail system and method may provide entertainment, facilitate the shopping experience, offer virtual coupons, render games based on locations throughout a store or based on a shopping list, provide information about food choices such as calorie counts, and identify metadata associated with items.

First claim

What is claimed is:

1. A method for generating a retail experience, comprising:
recognizing, by an electronic Augmented Reality (AR) system, a location of a user in a retail establishment;
said AR system retrieving data corresponding to the retail establishment;
said AR system generating virtual content relating to the retail establishment based on the retrieved data;
said AR system creating and rendering a virtual user interface in a field of view of the user and positioning the virtual user interface by tying the virtual user interface to object map points associated with a physical shopping cart in a real world retail location;
displaying the virtual content on the virtual user interface of a display device of the AR system, while the user is engaged in retail activity in the retail establishment;
said AR system moving the virtual user interface in response to a movement of the physical shopping cart in the real world retail location by rendering the virtual user interface at a position that is tied to the object map points associated with the physical shopping cart in the real world retail location after movement thereof such that the position of the virtual user interface relative to the physical shopping cart in the real world retail location remains unchanged;
receiving user input in a form of a user gesture; and
said AR system generating additional virtual content responsive to the user gesture,
wherein the user gesture comprises hand motions or hand positions and does not involve a user's hand contacting a physical interface or physical input device.

2. The method of claim 1 , wherein the retrieved data comprises a set of map points corresponding to the retail establishment.

3. The method of claim 1 , wherein the location of the user is recognized using radio frequency identification transponders and communications.

4. The method of claim 1 , wherein the virtual content is selected from the group consisting of a virtual character, a virtual coupon, a game based on the user location, a list of promotional items, nutritional information, metadata relating to an item, a celebrity appearance, a cross-selling advertisement, information from a person known to the user, and an electronic book.

5. The method of claim 1 , further comprising:
retrieving user data corresponding to the user; and
generating the virtual content based on both the retrieved data and the retrieved user data.

6. The method of claim 5 , wherein the virtual content is selected from the group consisting of a virtual grocery list, a virtual coupon book, a virtual recipe book, and a list of various ingredients in the user's home.

7. The method of claim 1 , further comprising:
receiving further user input;
generating further virtual content based on the further user input; and
displaying the further virtual content on the virtual user interface on a display device of the AR system, while the user is engaged in the retail activity in the retail establishment.

8. The method of claim 7 , wherein the further user input is selected from the group consisting of visual data, audio data, sensory data, a direct command, a voice command, eye tracking, and selection of a physical button.

9. The method of claim 5 , wherein the virtual content is selected from the group consisting of a running total cost, a smart virtual grocery list, an indicator of items proximate the user location, and a virtual payment system.

10. The method of claim 1 , further comprising sending the generated virtual content to the display device and wherein the AR system comprises a head mounted AR system and said creating a virtual user interface comprises creating the virtual user interface in the field of view of the user wearing the head mounted AR system.

11. The method of claim 1 implemented as a system with a processor capable of implementing the method steps.

12. The method of claim 1 implemented as a computer program product comprising a computer-readable storage medium having executable code to execute the method steps.

13. The method of claim 1 , further comprising comparing said gesture to a database to detect a predetermined command.

14. The method of claim 1 , further comprising detecting a location of a user by detecting a set of data associated with map points of the real world retail location and wherein said recognizing includes recognizing that the location of a user is in the retail establishment, based on the set of data.

15. The method of claim 1 , wherein the virtual content includes metadata relating to an item and further comprising unlocking the metadata in response to the gestures.

16. The method of claim 1 , wherein the display device includes artificial or remote image data superimposed, overlaid, or juxtaposed with real scenes.

17. The method of claim 1 , wherein the virtual user interface comprises a floating virtual user interface.

18. The method of claim 17 , wherein the virtual content comprises an animated virtual character.

19. The method of claim 14 , further comprising:
concluding the retail experience in response to detecting that the user has crossed a location threshold, wherein a conclusion of the retail experience corresponds to at least one of totaling up a cost of grocery items detected in the physical shopping cart, completing a checkout process through the virtual user interface, or identifying any items on a virtual grocery list that are not detected in the physical shopping cart.

20. The method of claim 1 , wherein the additional virtual content further comprises rendering a virtual effect to the user by recognizing a visual input and retrieving data corresponding to a virtually augmented appearance of the user, wherein the virtually augmented appearance is based at least in part on a diet the user is following.

Assignees

• Magic Leap Inc

Inventors

• Abovitz Rony
• Schowengerdt Brian T
• Watson Mathew D

Classifications

• G01B11/303 Primary

  using photoelectric detection means · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/758

  Involving statistics of pixels or of feature values, e.g. histogram matching · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title