Predicting household demographics based on image data

Patent metadata

Field	Value
Publication number	US-10277714-B2
Application number	US-201715592108-A
Country	US
Kind code	B2
Filing date	May 10, 2017
Priority date	May 10, 2017
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

An online system predicts household features of a user, e.g., household size and demographic composition, based on image data of the user, e.g., profile photos, photos posted by the user and photos posted by other users socially connected with the user, and textual data in the user's profile that suggests relationships among individuals shown in the image data of the user. The online system applies one or more models trained using deep learning techniques to generate the predictions. For example, a trained image analysis model identifies each individual depicted in the photos of the user; a trained text analysis model derive household member relationship information from the user's profile data and tags associated with the photos. The online system uses the predictions to build more information about the user and his/her household in the online system, and provide improved and targeted content delivery to the user and the user's household.

First claim

What is claimed is:

1. A method comprising:
receiving one or more photos associated with a user of an online system, each of the received photos including a plurality of visual features associated with each individual in the received photo;
receiving textual information describing the user and the one or more photos associated with the user;
applying a trained image analysis model to the received one or more photos to identify the plurality of visual features associated with each individual in the received photos;
applying a trained text analysis model to the textual information describing the user and the one or more photos to generate a plurality of textual features related to household features of the user;
generating one or more predictions of the household features of the user based on the plurality of visual features associated with each individual in the received one or more photos and the textual features related to the household features of the user;
evaluating a prediction of a number of members associated with a household of the user based on information describing a plurality of household devices and corresponding household device users associated with the user; and
storing the generated predictions associated with a profile of the user within the online system, the stored predictions used in targeting content to the user and to other members of the user's household.

2. The method of claim 1 , wherein the household features of the user comprise size of the user's household and demographic composition of the user's household.

3. The method of claim 1 , wherein storing the generated predictions associated with a profile of the user within the online system comprises:
supplementing the profile of the user within the online system with the predicted household features of the user; and
generating a comprehensive profile of the user based on the supplementing of the profile of the user.

4. The method of claim 3 , further comprising:
sharing the comprehensive profile of the user within the online system with another online system where the user has another user profile.

5. The method of claim 1 , further comprising:
selecting one or more content items for display to the user on the online system based on the predictions of the household features of the user; and
displaying the selected one or more content items to the user on the online system.

6. The method of claim 1 , wherein the image analysis model is trained based on a machine learning scheme associated with image processing on a corpus of image training data.

7. The method of claim 1 , wherein the text analysis model is trained based on a machine learning scheme associated with natural language processing on a corpus of textual training data.

8. The method of claim 1 , further comprising:
receiving information describing a plurality of household devices and corresponding household device users associated with the user; and
applying the trained text analysis model to the received information to generate information describing the household features of the user.

9. The method of claim 8 , wherein the information describing the household features of the user includes
an Internet Protocol address shared by the plurality of household devices of user, the method further comprising determining at least one of a size and a demographic composition of the household of the user based on the plurality of visual features, the plurality of textual features, and the internet protocol address shared by the plurality of household devices.

10. The method of claim 1 , further comprising reevaluating a prediction of a number of members residing in a household of the user based on information describing a plurality of household devices and corresponding household device users associated with the user.

11. A non-transitory computer-readable medium comprising computer program instructions, the computer program instructions when executed by a processor of a computer device causes the processor to perform the steps including:
receiving one or more photos associated with a user of an online system, each of the received photos including a plurality of visual features associated with each individual in the received photo;
receiving textual information describing the user and the one or more photos associated with the user;
applying a trained image analysis model to the received one or more photos to identify the plurality of visual features associated with each individual in the received photos;
applying a trained text analysis model to the textual information describing the user and the one or more photos to generate a plurality of textual features related to household features of the user;
generating one or more predictions of the household features of the user based on the plurality of visual features associated with each individual in the received one or more photos and the textual features related to the household features of the user;
evaluating a prediction of a number of members associated with a household of the user based on information describing a plurality of household devices and corresponding household device users associated with the user; and
storing the generated predictions associated with a profile of the user within the online system, the stored predictions used in targeting content to the user and to other members of the user's household.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the household features of the user comprise size of the user's household and demographic composition of the user's household.

13. The non-transitory computer-readable storage medium of claim 11 , wherein storing the generated predictions associated with a profile of the user within the online system comprises:
supplementing the profile of the user within the online system with the predicted household features of the user; and
generating a comprehensive profile of the user based on the supplementing of the profile of the user.

14. The non-transitory computer-readable storage medium of claim 13 , further comprising computer program instructions, the computer program instructions when executed by the processor of the computer device causes the processor to perform the steps including:
sharing the comprehensive profile of the user within the online system with another online system where the user has another user profile.

15. The non-transitory computer-readable storage medium of claim 11 , further comprising computer program instructions, the computer program instructions when executed by the processor of the computer device causes the processor to perform the steps including:
selecting one or more content items for display to the user on the online system based on the predictions of the household features of the user; and
displaying the selected one or more content items to the user on the online system.

16. The non-transitory computer-readable storage medium of claim 11 , wherein the image analysis model is trained based on a machine learning scheme associated with image processing on a corpus of image training data.

17. The non-transitory computer-readable storage medium of claim 11 , wherein the text analysis model is trained based on a machine learning scheme associated with natural language processing on a corpus of textual training data.

18. The non-transitory computer-readable storage medium of claim 11 , further comprising computer program instructions, the computer program instructions when executed by the processor of the computer device causes the

Assignees

• Facebook Inc

Inventors

• Bullock William
• Xu Liang
• Zhou Li

Classifications

• H04L67/63

  Routing a service request depending on the request content or context · CPC title

• H04L67/306 Primary

  User profiles · CPC title

• G06Q30/0269

  based on user profile or attribute · CPC title

• G06Q30/02

  Marketing; Price estimation or determination; Fundraising · CPC title

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title