Generating data in a messaging system for a machine learning model

Patent metadata

Field	Value
Publication number	US-10643104-B1
Application number	US-201715829572-A
Country	US
Kind code	B1
Filing date	Dec 1, 2017
Priority date	Dec 1, 2017
Publication date	May 5, 2020
Grant date	May 5, 2020

Abstract

Systems and methods are provided for analyzing location data associated with a location of a computing device to determine that a media content item is captured near a food-related venue or event, presenting interactive features to capture input related to food associated with the food-related venue or event, receiving the input in response to the presented interactive features, sending the media content item and the input in response to the interactive features to a computing system to incorporate the media content item and input into a machine learning model for food detection, and updating a messaging application to update a food detector functionality of the messaging application to comprise an updated machine learning model for food detection based on the media content item and input in response to the interactive features.

First claim

What is claimed is:

1. A method comprising:
detecting, by a computing device, capture of an image or video by the computing device using a messaging application;
analyzing, by the computing device, location data associated with a location of the computing device to determine that the image or video is captured near a food-related venue or event;
presenting, by the computing device, interactive features on a display of the computing device to capture an input to the computing device related to food associated with the food-related venue or event;
receiving, by the computing device, the input related to food associated with the food-related venue or event in response to the presented interactive features;
sending, by the computing device, the image or video and the input related to food associated with the food-related venue or event in response to the interactive features, to a computing system to incorporate the image or video and input into a machine learning model for food detection; and
updating, by the computing device, the messaging application to update a food detector functionality of the messaging application to comprise an updated machine learning model for food detection based on the image or video and the input related to food associated with the food-related venue or event in response to the interactive features.

2. The method of claim 1 , wherein after analyzing the location data associated with the location of the computing device to determine that the image or video is captured near the food-related venue or event, the method further comprises:
determining that the image or video comprises food using the machine learning model for food detection.

3. The method of claim 2 , wherein determining that the image or video comprises food using the machine learning model for food detection comprises:
inputting the image or video into the machine learning model for food detection;
generating a prediction value for the image or video;
determining whether the prediction value exceeds a predetermined threshold value; and
determining that the image or video comprises food based on the prediction value exceeding the predetermined threshold value.

4. The method of claim 1 , wherein analyzing the location data associated with the location of the computing device to determine that the image or video is captured near the food-related venue or event comprises comparing the location data to map data to determine that the location of the computing device is near the food-related venue or event.

5. The method of claim 1 , wherein the interactive features comprise a media overlay, a lens, or a request for a review of the food-related venue or event.

6. The method of claim 1 , wherein the interactive features comprise a request for an image of food from the food-related venue or event, and the image or video sent to the computing system is the requested image.

7. The method of claim 1 , wherein the computing system receives the image or video and associated input from the computing device and stores the received image or video and associated input in a database with a plurality of captured images and videos and associated input from a plurality of computing devices.

8. The method of claim 1 , wherein the updated machine learning model for food detection based on the image or video and the input in response to the interactive features is further based on a plurality of captured images and videos and input from a plurality of users in response to interactive features sent by a plurality of computing devices.

9. The method of claim 1 , wherein after analyzing the location data associated with the location of the computing device to determine that the image or video is captured near the food-related venue or event, the method further comprises:
determining that the image or video does not comprise food, using the trained machine learning model for food detection; and
wherein the interactive features presented comprise features to incentivize capture of an image of food associated with the food-related venue or event.

10. A computing device comprising:
one or more hardware processors; and
a computer-readable medium coupled with the one or more hardware processors, the computer-readable medium comprising instructions stored thereon that are executable by the one or more hardware processors to cause the computing device to perform operations comprising:
detecting capture of an image or video by the computing device using a messaging application;
analyzing location data associated with a location of the computing device to determine that the image or video is captured near a food-related venue or event;
presenting interactive features on a display of the computing device to capture an input to the computing device related to food associated with the food-related venue or event;
receiving the input related to food associated with the food-related venue or event in response to the presented interactive features;
sending the image or video and the input related to food associated with the food-related venue or event in response to the interactive features, to a computing system to incorporate the image or video and input into a machine learning model for food detection; and
updating the messaging application to update a food detector functionality of the messaging application to comprise an updated machine learning model for food detection based on the image or video and the input related to food associated with the food-related venue or event in response to the interactive features.

11. The computing device of claim 10 , wherein after analyzing the location data associated with the location of the computing device to determine that the image or video is captured near the food-related venue or event, the operations further comprise:
determining that the image or video comprises food using a machine learning model for food detection.

12. The computing device of claim 11 , wherein determining that the image or video comprises food using the trained machine learning model for food detection comprises:
inputting the image or video into the machine learning model for food detection;
generating a prediction value for the image or video;
determining whether the prediction value exceeds a predetermined threshold value; and
determining that the image or video comprises food based on the prediction value exceeding the predetermined threshold value.

13. The computing device of claim 10 , wherein analyzing the location data associated with the location of the computing device to determine that the image or video is captured near the food-related venue or event comprises comparing the location data to map data to determine that the location of the computing device is near the food-related venue or event.

14. The computing device of claim 10 , wherein the interactive features comprise a media overlay, a lens, or a request for a review of the food-related venue or event.

15. The computing device of claim 10 , wherein the interactive features comprise a request for an image of food from the food-related venue or event, and the image or video sent to the computing system is the requested image.

16. The computing device of claim 10 , wherein the computing system receives the image or video and associated input from the computing device and stores the received image or video and associated input in a database with a plurality of captured images and videos and associated input from a plurality of computing devices.

17. The computing device of claim 10 , wherein the updated machine learni

Assignees

• Snap Inc

Inventors

• Xue Zehao
• Ren Zhou

Classifications

• G06K9/6253 Primary

  Physics · mapped topic

• G06K2209/17

  Physics · mapped topic

• G06K2209/27

  Physics · mapped topic

• G06N20/00 Primary

  Machine learning · CPC title

• G06K9/6256

  Physics · mapped topic