System and method for augmented reality with complex augmented reality video image tags

Patent metadata

Field	Value
Publication number	US-9626807-B2
Application number	US-201615046764-A
Country	US
Kind code	B2
Filing date	Feb 18, 2016
Priority date	Nov 15, 2010
Publication date	Apr 18, 2017
Grant date	Apr 18, 2017

Abstract

Disclosed is a method for enabling use of a complex augmented reality system and a mobile device to display complex augmented reality on a display device. The method includes: enabling a user to capture one or more images from the display device via a camera on the mobile device; determining if there are one or more image tags within a video display on the display device that are viewable in the live camera image; using the one or more image tags to access information used by the augmented reality system; and overlaying virtual graphics in a position within 3D space on a display of the mobile device that augment the display of the display device with 3D effects to create complex augmented reality, when one or more image tags are detected within a video display on the display device that are viewable in a live camera image.

First claim

What is claimed is:

1. A method of enabling a user in an augmented reality gaming venue to use an augmented reality system to display complex augmented reality, the method comprising:
enabling the user to capture video of a gaming machine via a mobile device comprising a camera, wherein the video comprises one more images of a game or a game title;
sending the video via a network to a server;
determining if there are one or more image tags that are viewable in the video;
when one or more image tags are detected within the video:
using the one or more image tags to access information relating to the gaming machine;
transmitting virtual graphics over the network to the mobile device;
overlaying the virtual graphics in a position on a video display of the mobile device within 3D space to augment a video display of the gaming machine with 3D effects to create complex augmented reality.

2. The method of claim 1 , wherein the virtual graphics comprise one or more displayed controls by which the user may affect one or more operations of the gaming machine from the mobile device.

3. The method of claim 1 , wherein the video comprises an image of the video display of the gaming machine, and wherein any moving areas of the video are masked off to provide a remaining static image.

4. The method of claim 1 , further comprising: using a depth buffer to determine which elements of a 3D rendering are visible to a virtual camera.

5. The method of claim 4 , further comprising the step of comparing a z-value of each element with an existing depth buffer z-value at the same screen co-ordinate.

6. The method of claim 4 , further comprising: rendering the virtual gaming machine cabinet by mapping the image tag co-ordinates to the gaming machine cabinet co-ordinates using vector subtraction.

7. The method of claim 6 , further comprising: combining the camera image with the rendering of the additional virtual 3D component minus the virtual gaming machine cabinet.

8. The method of claim 1 , wherein a CAD model having spatial parameters is associated with an image tag.

9. The method of claim 1 , wherein the virtual rendering of the gaming machine cabinet is a virtual depth-only rendering of the gaming machine cabinet.

10. The method of claim 1 , wherein virtual rendering of the gaming machine cabinet extracts 2D information from the display of the gaming machine to produce a 3D image of the gaming machine.

11. The method of claim 1 , wherein an object recognition system of the augmented reality system establishes a communication channel with the gaming machine and the mobile device over the network so that additional elements of the game may be augmented on the mobile device and elements of the game content may be influenced by the augmented state.

12. The method of claim 1 , wherein a translation system of the augmented reality system extracts a first language from the display of the gaming machine and translates the first language into a second language that is visible to the user in place of the first language.

13. A method of enabling a user in an augmented reality gaming venue to use an augmented reality system to display complex augmented reality, the method comprising:
enabling a user to capture video data comprising one or more images of a first display device via a camera on a mobile device;
sending the video data via a network to a server;
determining if there are one or more image tags within the video data;
when one or more image tags are detected within the video data:
using the one or more image tags to access information used by the augmented reality system;
transmitting virtual graphics to the mobile device over the network;
overlaying the virtual graphics in a position within 3D space on a second display of the mobile device to augment the first display with 3D effects to create complex augmented reality.

14. The method of claim 13 , wherein at least one of the first display and the second display comprises an LCD screen.

15. The method of claim 13 , wherein any moving areas of the video data are masked off to provide a remaining static image.

16. The method of claim 13 , further comprising: using a depth buffer to determine which elements of a 3D rendering are visible to a virtual camera.

17. The method of claim 16 , further comprising comparing the z-value of each element to an existing depth buffer z-value at the same screen co-ordinate.

18. The method of claim 13 , wherein an object recognition system of the augmented reality system establishes a communication channel with the mobile device over the network so that additional elements of the game may be augmented on the mobile device and elements of the game content may be influenced by the augmented state.

19. The method of claim 13 , wherein a translation system of the augmented reality system extracts a first language from the first display and translates the first language into a second language that is visible to the user on the second display in place of the first language.

20. An augmented reality system to display complex augmented reality on a gaming machine, the system comprising:
a gaming machine comprising a first video display;
a mobile device comprising a camera and a second display;
a server;
a network connecting the server to the gaming machine and the mobile device; and
means for:
capturing one or more images of the gaming machine via the camera, wherein the images includes an image of a game or a game title;
sending the images of the gaming machine over the network to the server;
determining if there are one or more image tags within the images of the gaming machine;
when one or more image tags are detected, using the one or more image tags to access information relating to the gaming machine;
transmitting virtual graphics over the network to the mobile device; and
overlaying the virtual graphics in a position within 3D space on the second display to augment the first display with 3D effects to create complex augmented reality.

Assignees

• Bally Gaming Inc

Inventors

• Lyons Martin S
• Hedrick Joseph R
• Jackson Brett
• Ang Roderick

Classifications

• G06K9/00671

  Physics · mapped topic

• A63F13/426

  involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun · CPC title

• G06F3/14

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title

• A63F2009/2439

  the input being a code, e.g. ID · CPC title

• G06T2215/16

  Using real world measurements to influence rendering · CPC title