Modeling user interest in social media using news media and wikipedia

Abstract

Social media has become an important source of information and a medium for following and spreading trends, news, and ideas all over the world. Although determining the subjects of individual posts is important to extract users' interests from social media, this task is nontrivial because posts are highly contextualized and informal and have limited length. To address this problem, we propose a user modeling framework that maps the content of texts in social media to relevant categories in news media. In our framework, the semantic gaps between social media and news media are reduced by using Wikipedia as an external knowledge base. We map term-based features from a short text and a news category into Wikipedia-based features such as Wikipedia categories and article entities. A user's microposts are thus represented in a rich feature space of words. Experimental results show that our proposed method using Wikipedia-based features outperforms other existing methods of identifying users' interests from social media.

Introduction

Social media services such as Twitter and Facebook attract and encourage millions of users to share and exchange their ideas and opinions and to participate in events. Millions of new posts are generated daily from such open broadcasting platforms, and most of this information is stored in various text formats. Capturing users' interests from texts in social media data has become an important research topic in the area of personalized recommender systems. However, it is difficult to estimate the interests of social media users directly from social media data because their posts do not contain any category information [1]. To address this problem, Han and Lee [1] proposed an approach to map the contents of texts in social media into categories of a news corpus. Social media and news media are similar in that many current issues are posted in both. News media, however, contains additional information because news articles are categorized by experts into predefined categories. In Ref. [1], users' interests were estimated by comparing the features of news categories and features of personal social media data, where features were extracted from keywords in documents. This method is effective for categorization tasks, where the category of a social media post can easily be identified by distinguishable keywords such as “Obama” or “football”, but limitations still remain in dealing with short posts, abbreviations, and infrequently used topical terms. For instance, social media users often use the term “SNS” instead of the full name “social networking service”, whereas news media use the term “social media service”. Furthermore, terms that rarely occur in news media, such as the Indonesian dish “Nasi kuning”, do not provide any categorization information. We refer to this issue as the semantic gap between social media and news media.

To resolve this semantic gap, we employ Wikipedia as an external knowledge resource. Containing more than three million articles, Wikipedia is currently the world's largest knowledge resource. Each article describes a single topic with a succinct and well-formed title. Wikipedia also contains rich information about relationships between different articles in the forms of categories, interlinks, and redirect pages. A redirect page contains no content by itself; instead, it sends the user to another page, usually an article or a section of an article. For instance, searching for the keyword “UK” in Wikipedia results in an article page with the title “United Kingdom”. Thus, abbreviations in social media data can be resolved using Wikipedia. The problem caused by short texts or by infrequently occurring topical terms can be solved by enriching the terms using Wikipedia articles, categories, and their relationships. If we search for the term “Nasi kuning” in Wikipedia articles, we can extract not only the “Food” category but also other semantically related categories of the article.

In this study, we propose a new method to estimate users' interest in social media by mapping social media content into news categories. In this way, features generated using Wikipedia are used to resolve the semantic gap between social media data and news media data. Our proposed Wikipedia-based feature generators consist of the following three components: (i) Wiki-CF-ICF (Wikipedia-category frequency-inverse category frequency), which exploits the category information of Wikipedia articles; (ii) Wiki-AF-IAF (Wikipedia-article frequency-inverse article frequency), which uses the contents of Wikipedia articles; and (iii) Wiki-Cluster, which clusters features from Wiki-CF-ICF to filter out noise categories. Users' interests are estimated by calculating the similarity of features of a social media user and features of a news category. In the proposed approach, we regard each post of a user as a document instead of using an author-pool approach that generates a document for each user by aggregating all messages posted [1]. We evaluated the proposed approach by measuring the similarity score between a user's interests as estimated by our approach and as manually labeled by annotators. The evaluation results show that a mixture of Wikipedia-based approaches outperformed other approaches, such as term frequency-inverse category frequency (TF-ICF) [1] and Latent Dirichlet Allocation (LDA) [2], and that the approach of profiling users by regarding each post in social media as a document estimates users' interests more accurately than the author-pool approach. The remainder of this paper is organized as follows: Section 2 describes related works. Section 3 presents the framework of the proposed method. Section 4 describes the methodology for modeling user interest. Section 5 presents the experimental results. Section 6 presents the conclusions of the study.