Predicting topics of potential relevance based on retrieved/created digital media files

Patent metadata

Field	Value
Publication number	US-10860642-B2
Application number	US-201816014730-A
Country	US
Kind code	B2
Filing date	Jun 21, 2018
Priority date	Jun 21, 2018
Publication date	Dec 8, 2020
Grant date	Dec 8, 2020

Abstract

Implementations are described herein for leveraging digital media files retrieved and/or created by users to predict/determine topics of potential relevance to the users. In various implementations, digital media file(s) created and/or retrieved by a user with a client device may be applied as input across trained machine learning model(s), which in some cases are local to the client device, to generate output that indicates object(s) detected in the digital media file(s). Data indicative of the indicated object(s) may be provided to a remote computing system without providing the digital media file(s) themselves. In some implementations, information associated with the indicated object(s) may be retrieved and proactively output to the user. In some implementations, a frequency at which objects occur across a corpus of digital media files may be considered when determining a likelihood that a detected object is potentially relevant to a user.

First claim

What is claimed is:

1. A method implemented using one or more processors, comprising:
receiving data indicative of one or more digital images from one or more client devices operated by a user;
identifying one or more objects detected in each of the one or more digital images by performing image recognition processing of the data indicative of the one or more digital images by one or more of the processors;
determining a frequency at which each of the one or more objects occurs across a corpus of digital images provided by a population of users;
identifying one or more topics of potential relevance to the user based on the frequencies at which the one or more objects occur across the corpus, wherein a frequency at which a given object occurs across the corpus is inversely related to a likelihood a topic is identified from the object;
formulating a search query based on the identified one or more topics; and
proactively causing information responsive to the formulated search query to be output at one or more of the client devices operated by the user.

2. The method of claim 1 , wherein the data indicative of the one or more digital images comprises one or more feature vectors or reduced dimensionality embeddings extracted from the one or more digital images.

3. The method of claim 1 , wherein the one or more topics of potential relevance to the user are further identified based on positions of the one or more objects in the one or more digital images.

4. The method of claim 1 , wherein the one or more topics of potential relevance to the user are further identified based on measures of focus of the one or more objects in the one or more digital images.

5. The method of claim 1 , wherein the one or more topics of potential relevance to the user are further identified based on whether the user created or retrieved the one or more digital images received from the one or more client devices.

6. A system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to:
receive data indicative of one or more digital images from one or more client devices operated by a user;
identify one or more objects detected in each of the one or more digital images by performing image processing of the data indicative of the one or more digital images by one or more of the processors;
determine a frequency at which each of the one or more objects occurs across a corpus of digital images provided by a population of users;
identify one or more topics of potential relevance to the user based on the frequencies of the one or more objects across the corpus, wherein a frequency of a given object across the corpus is inversely related to a likelihood a topic is identified from the object;
formulate a search query based on the identified one or more topics; and
proactively cause information responsive to the formulated search query to be output at one or more of the client devices operated by the user.

7. The system of claim 6 , wherein the data indicative of the one or more digital images comprises one or more feature vectors or reduced dimensionality embeddings extracted from the one or more digital images.

8. The system of claim 6 , wherein the one or more topics of potential relevance to the user are further identified based on positions of the one or more objects in the one or more digital images.

9. The system of claim 6 , wherein the one or more topics of potential relevance to the user are further identified based on measures of focus of the one or more objects in the one or more digital images.

10. The system of claim 6 , wherein the one or more topics of potential relevance to the user are further identified based on whether the user created or retrieved the one or more digital images received from the one or more client devices.

11. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to:
receive data indicative of one or more digital images from one or more client devices operated by a user;
identify one or more objects detected in each of the one or more digital images based on the data indicative of the one or more digital images;
determine a frequency at which each of the one or more objects occurs across a corpus of digital images provided by a population of users;
identify one or more topics of potential relevance to the user based on the frequencies of the one or more objects across the corpus, wherein a frequency of a given object across the corpus is inversely related to a likelihood a topic is identified from the object;
formulate a search query based on the identified one or more topics; and
proactively cause information responsive to the formulated search query to be output at one or more of the client devices operated by the user.

12. The at least one non-transitory computer-readable medium of claim 11 , wherein the one or more topics of potential relevance to the user are further identified based on measures of focus of the one or more objects in the one or more digital images.

13. The method of claim 1 , wherein the determining includes applying term frequency-inverse document frequency (“TF-IDF”).

14. The system of claim 1 , wherein the frequency at which each of the one or more objects occurs across the corpus is determined via application of term frequency-inverse document frequency (“TF-IDF”).

15. The at least one non-transitory computer-readable medium of claim 11 , wherein the frequency at which each of the one or more objects occurs across the corpus is determined via application of term frequency-inverse document frequency (“TF-IDF”).

Assignees

• Google Llc

Inventors

• Rose Robert
• Cao Qun

Classifications

• G06F16/144

  Query formulation · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/908 Primary

  using metadata automatically derived from the content · CPC title

• G06F16/9032

  Query formulation · CPC title

• G06F16/5838 Primary

  using colour · CPC title