Predicting topics of potential relevance based on retrieved/created digital media files

Patent metadata

Field	Value
Publication number	US-11971925-B2
Application number	US-202318094245-A
Country	US
Kind code	B2
Filing date	Jan 6, 2023
Priority date	Jun 21, 2018
Publication date	Apr 30, 2024
Grant date	Apr 30, 2024

Abstract

Implementations are described herein for leveraging digital media files retrieved and/or created by users to predict/determine topics of potential relevance to the users. In various implementations, digital media file(s) created and/or retrieved by a user with a client device may be applied as input across trained machine learning model(s), which in some cases are local to the client device, to generate output that indicates object(s) detected in the digital media file(s). Data indicative of the indicated object(s) may be provided to a remote computing system without providing the digital media file(s) themselves. In some implementations, information associated with the indicated object(s) may be retrieved and proactively output to the user. In some implementations, a frequency at which objects occur across a corpus of digital media files may be considered when determining a likelihood that a detected object is potentially relevant to a user.

First claim

What is claimed is:

1. A method implemented using one or more processors, comprising:
obtaining one or more digital image files accessed by a user with one or more client devices;
applying the one or more digital image files as input across one or more trained object recognition machine learning models, wherein the one or more trained object recognition machine learning models generate output indicative of a plurality of objects detected in the one or more digital image files;
based on the plurality of detected objects, and using a topic hierarchy, identifying a genus that captures two or more detected objects within the plurality, wherein the identifying is further based on one or more measures of focus at which the two or more detected objects within the plurality are depicted in the one or more digital image files, wherein the one or more measures of focus indicate how blurry the two or more detected objects are compared to one or more other detected objects in the one or more digital image files;
identifying the genus as a topic of potential relevance to the user;
storing an association between the topic of potential relevance to the user and predetermined content contained in a database; and
based on the stored association, curating the predetermined content for presentation as output at one or more of the client devices operated by the user.

2. The method of claim 1 , wherein the genus comprises a type of location to which the two or more detected objects within the plurality are related.

3. The method of claim 2 , wherein the two or more detected objects within the plurality are related to the type of location by virtue of the detected objects being commonly found in the type of location.

4. The method of claim 1 , wherein the genus comprises a location in which the two or more detected objects within the plurality are detected.

5. The method of claim 1 , wherein one or more of the trained object recognition machine learning models comprises a convolutional neural network.

6. The method of claim 1 , wherein the genus is identified further based on a measure of prominence at which the two or more detected objects within the plurality are depicted.

7. A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
obtain one or more digital image files accessed by a user with one or more client devices;
apply the one or more digital image files as input across one or more trained object recognition machine learning models, wherein the one or more trained object recognition machine learning models generate output indicative of a plurality of objects detected in the one or more digital image files;
based on the plurality of detected objects, and using a topic hierarchy, identify a genus that captures two or more detected objects within the plurality, wherein the genus is further identified based on one or more measures of focus at which the two or more detected objects within the plurality are depicted in the one or more digital image files, wherein the one or more measures of focus indicate how blurry the two or more detected objects are compared to one or more other detected objects in the one or more digital image files;
identify the genus as a topic of potential relevance to the user;
store an association between the topic of potential relevance to the user and predetermined content contained in a database; and
based on the identified stored association, curate the predetermined content for presentation as output at one or more of the client devices operated by the user.

8. The system of claim 7 , wherein the genus comprises a type of location to which the two or more detected objects within the plurality are related.

9. The system of claim 8 , wherein the two or more detected objects within the plurality are related to the type of location by virtue of the detected objects of the subset being commonly found in the type of location.

10. The system of claim 7 , wherein the genus comprises a location in which the two or more detected objects within the plurality are detected.

11. The system of claim 7 , wherein one or more of the trained object recognition machine learning models comprises a convolutional neural network.

12. The system of claim 7 , wherein the genus is identified further based on a measure of prominence at which the two or more detected objects within the plurality are depicted.

13. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to:
obtain one or more digital image files accessed by a user with one or more client devices;
apply the one or more digital image files as input across one or more trained object recognition machine learning models, wherein the one or more trained object recognition machine learning models generate output indicative of a plurality of objects detected in the one or more digital image files;
based on the plurality of detected objects, and using a topic hierarchy, identify a genus that captures two or more detected objects within the plurality, wherein the genus is further identified based on one or more measures of focus at which the two or more detected objects within the plurality are depicted in the one or more digital image files, wherein the one or more measures of focus indicate how blurry the two or more detected objects are compared to one or more other detected objects in the one or more digital image files;
identify the genus as a topic of potential relevance to the user;
store an association between the topic of potential relevance to the user and predetermined content contained in a database; and
based on the identified stored association, curate the predetermined content for presentation as output at one or more of the client devices operated by the user.

14. The at least one non-transitory computer-readable medium of claim 13 , wherein the genus comprises a type of location to which the two or more detected objects within the plurality are related.

15. The at least one non-transitory computer-readable medium of claim 14 , wherein the two or more detected objects within the plurality are related to the type of location by virtue of the detected objects being commonly found in the type of location.

16. The at least one non-transitory computer-readable medium of claim 13 , wherein the genus comprises a location in which the two or more detected objects within the plurality are detected.

Assignees

• Google Llc

Inventors

• Rose Robert
• Cao Qun

Classifications

• G06F16/5838 Primary

  using colour · CPC title

• G06F16/144

  Query formulation · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/908 Primary

  using metadata automatically derived from the content · CPC title

• G06F16/9032

  Query formulation · CPC title