Michele Merler - Research Staff Member in the Multimedia Analytics group at the IBM TJ Watson Research Center in New York Multimedia Communications
Date: - Location: Eurecom
Deriving user attributes from social media has been an active area of research since the inception of social networks. Traditional methods focus on text and social linkage analysis, while discarding visual information contained in images and videos. On the other hand, recent years have witnessed the explosive rise of principally visual social media, such as Instagram and Pinterest, where textual information is extremely scarce and mostly unusable. In order to bridge this gap we introduce System V, a method to extract user attributes from the visual information in social media feeds. System V takes any public social media user handle as input and accesses the visual information in the social media feed (currently Instagram, Twitter, Pinterest, Tumblr, Youtube and Flickr are supported). It then applies to the images and videos a set of visual classifiers which were pre-trained using the IBM Multimedia Analysis and Retrieval System (IMARS), are related to family, leisure, pets, activities, life events, work and travel, and are organized in a semantic taxonomy. Classifiers score aggregation is performed in order to study the distribution of categories across all the visual information of the user. Finally, the system provides an interactive summary of a user (or a group of users) in the form of a web based radial treemap graph.
