Article Preview
TopIntroduction
Social Media services have become a very popular medium of communication and users use these services for various different reasons. In the case of Twitter, a microblogging service, the main reasons found are (Java, Song, Finin, & Tseng, 2007): daily chatter, conversations, sharing information and reporting news. Microblogging services in particular have become very popular due to their portability, immediacy and ease of use, allowing users to respond and spread information more rapidly (Atefeh & Khreich, 2015). The popularity and real time nature of these services and the fact that the data generated reflect aspects of real-world societies and is publicly available have attracted the attention of researchers in several fields (Madani, Boussaid, & Zegour, 2014; Nicolaos, Ioannis, & Dimitrios, 2016). One such field is the field of event detection in Social Media.
Event detection in Social Media has many potential applications, some of which with significant social impact such as in the detection of natural disasters and to identify and track diseases and epidemics (Madani et al., 2014). Another relevant application can be found in the detection of news topics and events of interest or newsworthy, as real-world events are often discussed by users in these services before they are even reported in traditional Media (Papadopoulos, Corney, & Aiello, 2014; Sakaki, Okazaki, & Matsuo, 2010; Van Canneyt et al., 2014). These services however present some challenges, some of which are inherit to their design and usage (Atefeh & Khreich, 2015). In the case of Twitter, the use of informal and abbreviated words, the frequent occurrence of spelling and grammatical errors, data sparseness and lack of context due to the short length of the messages, are just a few examples of these challenges. The diversity and nature of the topics discussed may also pose additional challenges, more specifically in the case of event detection, as most of these topics are of little interest (e.g. daily chatter). The event detection process must therefore be able to filter out these topics in order to retain only those potentially related to events of interest.
The goal of this work and also its main contribution is the implementation of a fully functional system in order to detect newsworthy events using tweets, that is, any real-world event of sufficient interest to the general public in order to be reported by the Media. To achieve this goal a similar methodology already proposed in the literature, namely Twevent (C. Li, Sun, & Datta, 2012) is used as the base of the implementation. The event detection pipeline proposed consists of the following steps: first the tweets are segmented into a set of non-overlapping segments (i.e. n-grams). These segments are then ranked according to a weighting scheme. Only the top K segments are retained for further processing, thus obtaining the event segments. A variant of the Jarvis-Patrick clustering algorithm is then used in order to cluster these event segments into candidate event clusters. Finally, these candidate event clusters are filtered in order to retain only those related to real world events of interest. This filtering step is performed by a Random Forest model.