Individual and user group attributes discovery and comparison from social media visual content

Patent metadata

Field	Value
Publication number	US-10282677-B2
Application number	US-201514933842-A
Country	US
Kind code	B2
Filing date	Nov 5, 2015
Priority date	Nov 5, 2015
Publication date	May 7, 2019
Grant date	May 7, 2019

Abstract

A method and system are provided. The method includes deriving a set of user attributes from an aggregate analysis of images and videos of a user. The deriving step includes recognizing, by a set of visual classifiers, semantic concepts in the images and videos of the user to generate visual classifier scores. The deriving step further includes deriving, by a statistical aggregator, the set of user attributes. The set of user attributes are derived by mapping the visual classifier scores to a taxonomy of semantic categories to be recognized in visual content. The deriving step also includes displaying, by an interactive user interface having a display, attribute profiles for the attributes and comparisons of the attribute profiles.

First claim

What is claimed is:

1. A method, comprising:
deriving a set of user attributes from an aggregate analysis of images and videos of a user by (i) recognizing, by a set of visual classifiers, semantic concepts in the images and videos of the user to generate visual classifier scores; and (ii) deriving, by a statistical aggregator, the set of user attributes by mapping the visual classifier scores to a taxonomy of semantic categories to be recognized in visual content; and
providing an interactive user interface, having at least one user input element and a display element, configured to interactively display attribute profiles for the attributes and comparisons of the attribute profiles.

2. The method of claim 1 , wherein the comparisons of the attribute profiles are provided using a differential analyzer, the differential analyzer using one or more distribution comparison metrics to provide the comparisons of the attribute profiles.

3. The method of claim 2 , wherein distributions of the visual classifier scores between any of single users and groups of users are compared by the differential analyzer to provide the comparisons of the attribute profiles.

4. The method of claim 1 , further comprising training an attribute prediction system using one or more distributions of the visual classifier scores from multiple labeled users.

5. The method of claim 1 , wherein the aggregate analysis is performed over the images and videos from any of, one or more single users and one or more groups of users.

6. The method of claim 1 , wherein the aggregation analysis is performed over any of images and videos taken at, at least one of, different times and different locations.

7. The method of claim 1 , wherein the images and videos are accessed via at least one of a users' social media stream, a phone, a tablet, and a computer gallery.

8. The method of claim 1 , wherein the taxonomy of semantic categories is configured to model diverse topics.

9. The method of claim 1 , wherein the set of visual classifiers are learned from labeled data, wherein labels for the labeled data are provided by at least one of annotation, tagging, and crowd-sourcing.

10. The method of claim 1 , further comprising pre-training the set of visual classifiers on an image dataset of visual classes, independently from any of the user attributes.

11. A non-transitory article of manufacture tangibly embodying a computer readable program which when executed causes a computer to perform the steps of claim 1 .

12. A system, comprising:
an aggregate analyzer for deriving a set of user attributes from an aggregate analysis of images and videos of a user,
wherein said aggregate analyzer includes:
a set of visual classifiers for recognizing semantic concepts in the images and videos of the user to generate visual classifier scores; and
a statistical aggregator for deriving the set of user attributes, wherein the set of user attributes are derived by mapping the visual classifier scores to a taxonomy of semantic categories to be recognized in visual content; and

an interactive user interface having at least one user input element and a display element for interactively displaying attribute profiles for the attributes and comparisons of the attribute profiles.

13. The system of claim 12 , further comprising a differential analyzer for providing the comparisons of the attribute profiles, the differential analyzer using one or more distribution comparison metrics to provide the comparisons of the attribute profiles.

14. The system of claim 13 , wherein distributions of the visual classifier scores between any of single users and groups of users are compared by the differential analyzer to provide the comparisons of the attribute profiles.

15. The system of claim 12 , wherein the aggregate analysis is performed over the images and videos from any of, one or more single users and one or more groups of users.

16. The system of claim 12 , wherein the aggregation analysis is performed over any of images and videos taken at, at least one of, different times and different locations.

17. The system of claim 12 , wherein the images and videos are accessed via at least one of a users' social media stream, a phone, a tablet, and a computer gallery.

18. The system of claim 12 , wherein the taxonomy of semantic categories is configured to model diverse topics.

19. The system of claim 12 , wherein the set of visual classifiers are learned from labeled data, wherein labels for the labeled data are provided by at least one of annotation, tagging, and crowd-sourcing.

20. The system of claim 12 , wherein the set of visual classifiers are pre-trained on an image dataset of visual classes, independently from any of the user attributes.

Assignees

• IBM

Inventors

• Merler Michele
• Park Jae-Eun
• Smith John R
• Uceda-Sosa Rosario

Classifications

• G06N99/005 Primary

  Physics · mapped topic

• H04N21/00

  Selective content distribution, e.g. interactive television or video on demand [VOD] (real-time bi-directional transmission of motion video data H04N7/14 {; broadcast or conference over packet switching networks H04L12/18}) · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F16/55 Primary

  Clustering; Classification · CPC title