Device location based on machine learning classifications

Patent metadata

Field	Value
Publication number	US-9980100-B1
Application number	US-201715692990-A
Country	US
Kind code	B1
Filing date	Aug 31, 2017
Priority date	Aug 31, 2017
Publication date	May 22, 2018
Grant date	May 22, 2018

Abstract

A venue system of a client device can submit a location request to a server, which returns multiple venues that are near the client device. The client device can use one or more machine learning schemes (e.g., convolutional neural networks) to determine that the client device is located in one of specific venues of the possible venues. The venue system can further select imagery for presentation based on the venue selection. The presentation may be published as ephemeral message on a network platform.

First claim

What is claimed is:

1. A method comprising:
transmitting, using one or more processors of a client device, a request for venue data to one or more servers, the request comprising geolocation data generated by the client device;
receiving, from the one or more servers, a venue dataset comprising a plurality of venues, each venue of the plurality of venues associated with tags that describe the venue;
identifying an image generated by the client device;
classifying an object depicted in the image using a machine learning scheme;
selecting a venue from the venue dataset based at least in part on the classified object matching a tag associated with the venue;
selecting one or more display elements that are pre-associated with the selected venue; and
displaying, on the client device, a presentation comprising the one or more display elements and the image.

2. The method of claim 1 , further comprising:
determining environment data for the image using an additional machine learning scheme, the environment data describing whether the image is of an outside environment or inside environment.

3. The method of claim 2 , wherein the selection of the venue is further based at least in part on the environment data of the image matching a tag associated with the venue.

4. The method of claim 1 , wherein the image is part of an active video feed generated by the client device; and
the presentation displays the one or more display elements overlaid on the active video feed.

5. The method of claim 1 , wherein the plurality of venues are categorized, on the client device, into categories and subcategories, and the tags are metadata of the categories or subcategories.

6. The method of claim 5 , wherein the one or more display elements are categorized into the categories and subcategories, and the one or more display elements are pre-associated with the selected venue in that the one or more display elements and the selected elements share at least one of a same category or subcategory.

7. The method of claim 1 , wherein the one or more display elements include an avatar of a user of the client device.

8. The method of claim 1 , wherein the geolocation data includes global positioning system (GPS) data generated by a GPS sensor on the client device.

9. The method of claim 1 , wherein the venue dataset is a subset of a larger venue dataset that includes venues that are not near to a location indicated by the geolocation data of the client device.

10. The method of claim 1 , wherein the machine learning scheme comprises one or more neural networks trained to classify physical objects using an image set, the image set comprising images of a plurality of physical images.

11. The method of claim 2 , wherein the additional machine learning scheme comprises one or more neural networks trained to classify environments as being inside or outside, the one or more neural networks trained on an image set comprising images of a plurality of images of outside and inside environments.

12. The method of claim 1 , further comprising:
receiving, on the client device, an instruction to store the presentation.

13. The method of claim 12 , further comprising:
publishing, to a social network site, the presentation as an ephemeral message.

14. A system comprising:
one or more processors of a machine; and
a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
transmitting a request for venue data to one or more servers, the request comprising geolocation data generated by a client device;

receiving, from the one or more servers, a venue dataset comprising a plurality of venues, each venue of the plurality of venues associated with tags that describe the venue;
identifying an image generated by the client device;
classifying an object depicted in the image using a machine learning scheme;
selecting a venue from the venue dataset based at least in part on the classified object matching a tag associated with the venue;
selecting one or more display elements that are pre-associated with the selected venue; and
displaying, on the client device, a presentation comprising the one or more display elements and the image.

15. The system of claim 14 , the operations further comprising:
determining environment data for the image using an additional machine learning scheme, the environment data describing whether the image is of an outside environment or inside environment.

16. The system of claim 15 , wherein the selection of the venue is further based at least in part on the environment data of the image matching a tag associated with the venue.

17. The system of claim 14 , wherein the image is part of an active video feed generated by the client device; and
the presentation displays the one or more display elements overlaid on the active video feed.

18. The system of claim 14 , wherein the plurality of venues are categorized, on the client device, into categories and subcategories, and the tags are metadata of the categories or subcategories.

19. A machine-readable storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
transmitting a request for venue data to one or more servers, the request comprising geolocation data generated by the client device;

receiving, from the one or more servers, a venue dataset comprising a plurality of venues near, each venue of the plurality of venues associated with tags that describe the venue;
identifying an image generated by the client device;
classifying an object depicted in the image using a machine learning scheme;
selecting a venue from the venue dataset based at least in part on the classified object matching a tag associated with the venue;
selecting one or more display elements that are pre-associated with the selected venue; and
displaying, on the client device, a presentation comprising the one or more display elements and the image.

20. The machine-readable storage medium of claim 19 , the operations further comprising:
determining environment data for the image using an additional machine learning scheme, the environment data describing whether the image is of an outside environment or inside environment.

Assignees

• Snap Inc

Inventors

• Charlton Ebony James
• Hanumante Sumant
• Ren Zhou
• Sagar Dhritiman

Classifications

• G06V20/35

  Categorising the entire scene, e.g. birthday party or wedding scene · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• H04W4/029

  Location-based management or tracking services · CPC title