Contextual local image recognition dataset

Patent metadata

Field	Value
Publication number	US-9613462-B2
Application number	US-201514747043-A
Country	US
Kind code	B2
Filing date	Jun 23, 2015
Priority date	Mar 15, 2013
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

A contextual local image recognition module of a device retrieves a primary content dataset from a server and then generates and updates a contextual content dataset based on an image captured with the device. The device stores the primary content dataset and the contextual content dataset. The primary content dataset comprises a first set of images and corresponding virtual object models. The contextual content dataset comprises a second set of images and corresponding virtual object models retrieved from the server.

First claim

What is claimed is:

1. A server comprising:
a network communication interface configured to:
communicate with a plurality of devices, and
receive, from a first device from the plurality of devices, an image captured by an optical sensor of the first device;

one or more hardware processors configured to:
generate a primary content dataset comprising a first set of images and corresponding virtual content, the first set of images including a core set of images,
compare the image received from the first device with the first set of images,
based on the comparison, retrieve virtual content associated with the image in response to determining that the first set of images does not include the image received from the first device, and
communicate, via the network communication interface, to the first device, the virtual content associated with the image received from the first device, the first device being configured to update a contextual content dataset with the virtual content associated with the image received from the device; and

a storage device coupled to the one or more hardware processors, the storage device comprising the primary content dataset, a secondary content dataset corresponding to the contextual content dataset of the plurality of devices, and a third content dataset comprising virtual content not included in the primary and secondary content datasets.

2. The server of claim 1 , wherein the core set of images includes images that were most commonly captured by the plurality of devices.

3. The server of claim 1 , wherein the one or more hardware processors are further configured to:
provide the primary content dataset to the plurality of devices;
generate a corresponding secondary content dataset for each device based on images captured from the corresponding device; and
generate a corresponding third content dataset for each device based on usage conditions of the corresponding device.

4. The server of claim 1 , wherein the one or more hardware processors are further configured to:
determine usage conditions of the first device;
identify virtual content based on the usage conditions and not included in the primary and secondary content datasets; and
communicate the virtual content based on the usage conditions and not included in the primary and secondary content datasets to the device.

5. The server of claim 4 , wherein the one or more hardware processors are further configured to:
update the third content dataset of the first device with the virtual content based on the usage conditions and not included in the primary and secondary content datasets.

6. The server of claim 4 , wherein the usage conditions of the first device include at least one of a time, duration, location, and orientation of the device at the time of capture of the image.

7. The server of claim 4 , wherein the usage conditions of the first device include images from other users in a social network of the user of the device.

8. The server of claim 1 , wherein the first device includes an augmented reality application configured to display the virtual content in a display of the device.

9. The server of claim 8 , wherein the one or more hardware processors are further configured to:
upload the primary content dataset to the augmented reality application of the first device in response to detecting an initialization of the augmented reality application in the first device.

10. The server of claim 8 , wherein usage conditions of the first device include at least one of a time and location of the first device while the augmented reality application operates in the first device.

11. A method comprising:
receiving, from a first device from a plurality of devices, an image captured by an optical sensor of the first device;
generating, at a server, a primary content dataset comprising a first set of images and corresponding virtual content, the first set of images including a core set of images;
comparing the image received from the first device with the first set of images;
based on the comparison, retrieving virtual content associated with the image in response to determining that the first set of images does not include the image captured by the first device;
communicating, to the first device, the virtual content associated with the image received from the first device, the first device being configured to update a contextual content dataset with the virtual content associated with the image received from the device; and
storing, at the server, the primary content dataset, a secondary content dataset corresponding to the contextual content dataset of the plurality of devices, and a third content dataset comprising virtual content not included in the primary and secondary content datasets.

12. The method of claim 11 , wherein the core set of images includes images that were most commonly captured from the plurality of devices.

13. The method of claim 11 , further comprising:
providing the primary content dataset to the plurality of devices;
generating a corresponding secondary content dataset for each device based on images captured from the corresponding device; and
generating a corresponding third content dataset for each device based on usage conditions of the corresponding device.

14. The method of claim 11 , further comprising:
determining usage conditions of the first device;
identifying virtual content based on the usage conditions and not included in the primary and secondary content datasets; and
communicating the virtual content based on the usage conditions and not included in the primary and secondary content datasets to the first device.

15. The method of claim 14 , further comprising:
updating the third content dataset of the first device with the virtual content based on the usage conditions and not included in the primary and secondary content datasets.

16. The method of claim 14 , wherein the usage conditions of the first device include at least one of a time, duration, location, and orientation of the first device at the time of capture of the image.

17. The method of claim 14 , wherein the usage conditions of the first device include images from other users in a social network of the user of the first device.

18. The method of claim 11 , wherein the first device includes an augmented reality application configured to display the virtual content in a display of the first device.

19. The method of claim 18 , further comprising:
uploading the primary content dataset to the augmented reality application of the first device in response to detecting an initialization of the augmented reality application in the first device,
wherein usage conditions of the first device include at least one of a time and location of the first device while the augmented reality application operates in the first device.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a server, cause the server to perform operations comprising:
receiving, from a first device from a plurality of devices, an image captured by an optical sensor of the first device;
generating a primary content dataset comprising a first set images and corresponding virtual content, the first set of images including a core set of images;
comparing the image received from the first device with the first set of images;
based on the comparison, retrieving virtual content associated with the image in response to determining that the first set of images does not inclu

Assignees

• Daqri Llc

Inventors

• Mullins Brian

Classifications

• G06T1/0007

  Image acquisition · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F1/1686

  the I/O peripheral being an integrated camera · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2200/04

  involving 3D image data · CPC title