Wide area augmented reality location-based services

Patent metadata

Field	Value
Publication number	US-12008719-B2
Application number	US-202217587183-A
Country	US
Kind code	B2
Filing date	Jan 28, 2022
Priority date	Oct 17, 2013
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

Apparatus, methods and systems of providing AR content are disclosed. Embodiments of the inventive subject matter can obtain an initial map of an area, derive views of interest, obtain AR content objects associated with the views of interest, establish experience clusters and generate a tile map tessellated based on the experience clusters. A user device could be configured to obtain and instantiate at least some of the AR content objects based on at least one of a location and a recognition.

First claim

What is claimed is:

1. A device capable of rendering augmented reality (AR), the device comprising:
a plurality of sensors, including a camera and a location sensor;
a display;
a non-transitory computer readable memory storing software instructions; and
at least one processor coupled with the non-transitory computer readable memory, the plurality of sensors, and the display; and, upon execution of the software instructions, is configurable to:
obtain sensor data from at least one sensor wherein the sensor data corresponds to a real-time perspective of a user, and includes image data from the camera and a device location obtained from the location sensor;
obtain an area of interest via an area database based on the sensor data;
access an area tile map of the area of interest, the area tile map represented by a set of tile subareas that includes one or more tessellated tiles from a tessellated tile map;
identify a tile subarea from the set of tile subareas based at least in part on the device location relative to one or more locations of tile subareas from the set of tile subareas, wherein the identified tile subarea covers at least a portion of the area of interest, and wherein one or more tessellated tiles within the identified tile subarea are associated with one or more AR content objects;
populate the non-transitory computer readable memory with at least one of the one or more AR content objects associated with the one or more tessellated tiles within the identified tile subarea; and
render the at least one of the one or more AR content objects that is associated with the identified tile subarea on the display as a visual overlay of a real-world image generated from the image data.

2. The device of claim 1 , wherein the location sensor comprises at least one of the following location sensors: a global positioning system (GPS) sensor, an image sensor, and a wireless sensor.

3. The device of claim 1 , wherein the device location includes at least one of the following: a global positioning system (GPS) location, a wireless signal location, a depth of field location, and an image-based simultaneous location and mapping (SLAM) location.

4. The device of claim 1 , wherein the at least one sensor further includes one or more of the following: an accelerometer and a gyroscope.

5. The device of claim 1 , wherein the sensor data further includes at least one of device position data and device orientation data.

6. The device of claim 5 , wherein the at least one of the one or more AR content objects is rendered based on a view of interest, wherein the view of interest is derived from at least one of the device position data and the device orientation data.

7. The device of claim 1 , wherein the at least one of the one or more AR content objects comprises the visual overlay rendered as a graphic sprite or an animation for an interactive game experience.

8. The device of claim 7 , wherein the interactive game experience comprises the graphic sprite or the animation interacting with one or more physical elements in the real-world image.

9. The device of claim 1 , wherein the device comprises a smart phone or a tablet.

10. The device of claim 1 , wherein the area of interest comprises one of the following: a landmark, user-defined boundaries, state-defined boundaries, natural boundaries, a city, a country, a business, a shopping center, a restaurant, a coffee shop, a warehouse, a stadium, a wilderness area, a road, a garden, a zoo, an amusement park, a beach and a building.

11. The device of claim 1 , wherein the area of interest comprises an outdoor setting.

12. The device of claim 1 , wherein the device communicatively couples with the area database over a network.

13. The device of claim 1 , wherein the at least one of the one or more AR content objects is obtained from an AR content database.

14. The device of claim 13 , wherein the device is communicatively coupled with the AR content database over a network.

15. The device of claim 1 , wherein the at least one of the one or more AR content objects is rendered relative to a recognized object.

16. The device of claim 15 , wherein the recognized object represents an attachment point for the at least one of the one or more AR content objects.

17. The device of claim 16 , wherein the attachment point includes at least one of the following: a billboard, a wall, a floor, a plant, a sign, a logo, a landmark, a building, and a car.

18. A method of rendering augmented reality (AR) with a device, the method comprising:
obtaining sensor data from at least one sensor on the device, the at least one sensor including a location sensor, wherein the sensor data corresponds to a real-time perspective of a user, and includes a device location obtained from the location sensor and real-world image data;
obtaining an area of interest via an area database based on at least the device location within the sensor data;
accessing an area tile map of the area of interest, the area tile map represented by a set of tile subareas that includes one or more tessellated tiles from a tessellated tile map;
identifying a tile subarea from the set of tile subareas based at least in part on the device location relative to one or more locations of tile subareas from the set of tile subareas, wherein the identified tile subarea covers at least a portion of the area of interest, and wherein one or more tessellated tiles within the identified tile subarea are associated with one or more AR content objects from an AR content database communicatively coupled with the device over a network;
automatically populating a non-transitory computer readable memory of the device with at least one of the one or more AR content objects from the AR content database, wherein the at least one of the one or more AR content objects is associated with the one or more tessellated tiles within the identified tile subarea; and
rendering the at least one of the one or more AR content objects that is associated with the identified tile subarea on a display of the device based on the device location relative to the area of interest, wherein the at least one of the one or more AR content objects is rendered as a visual overlay of a view of interest associated with the real-world image data.

19. The method of claim 18 , wherein the at least one of the one or more AR content objects comprises a game object.

20. The method of claim 18 , wherein the at least one of the one or more AR content objects comprises an interactive game experience.

21. The method of claim 18 , wherein the at least one of the one or more AR content objects comprises the visual overlay rendered as a graphic sprite or an animation.

22. The method of claim 18 , wherein the at least one of the one or more AR content objects comprises an exclusive content related to the identified tile subarea.

23. The method of claim 22 , wherein the exclusive content related to the identified tile subarea is rendered based on a paid fee.

24. The method of claim 18 , wherein the non-transitory computer readable memory, an at least one processor, the at least one sensor, and the display comprise at least one of the following: a smart phone, augmented reality goggles, a watch, a tablet, a drone, a vehicle, a wearable augmented reality device, and a robot.

25. The method of claim 18 , wherein the at least one of the one or more AR content objects is determined based on movement of th

Assignees

• Nant Holdings Ip Llc

Inventors

• Mckinnon David
• Wnuk Kamil
• Sudol Jeremi
• Siddiqui Matheen
• Wiacek John
• Song Bing
• Witchey Nicholas J

Classifications

• G06T2219/024

  Multi-user, collaborative environment · CPC title

• G06T15/20

  Perspective computation · CPC title

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title

• G06F16/5854

  using shape and object relationship · CPC title

• G06F16/58 Primary

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title