Manchester Metropolitan University's Research Repository

Sentiment analysis of tweets through Altmetrics: A machine learning approach

Hassan, Saeed-Ul and Saleem, Aneela and Soroya, Saira Hanif and Safder, Iqra and Iqbal, Sehrish and Jamil, Saqib and Bukhari, Faisal and Aljohani, Naif Radi and Nawaz, Raheel (2020) Sentiment analysis of tweets through Altmetrics: A machine learning approach. Journal of Information Science. 016555152093091-016555152093091. ISSN 0165-5515


Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (542kB) | Preview


The purpose of the study is to (a) contribute to annotating an Altmetrics dataset across five disciplines, (b) undertake sentiment analysis using various machine learning and natural language processing–based algorithms, (c) identify the best-performing model and (d) provide a Python library for sentiment analysis of an Altmetrics dataset. First, the researchers gave a set of guidelines to two human annotators familiar with the task of related tweet annotation of scientific literature. They duly labelled the sentiments, achieving an inter-annotator agreement (IAA) of 0.80 (Cohen’s Kappa). Then, the same experiments were run on two versions of the dataset: one with tweets in English and the other with tweets in 23 languages, including English. Using 6388 tweets about 300 papers indexed in Web of Science, the effectiveness of employed machine learning and natural language processing models was measured by comparing with well-known sentiment analysis models, that is, SentiStrength and Sentiment140, as the baseline. It was proved that Support Vector Machine with uni-gram outperformed all the other classifiers and baseline methods employed, with an accuracy of over 85%, followed by Logistic Regression at 83% accuracy and Naïve Bayes at 80%. The precision, recall and F1 scores for Support Vector Machine, Logistic Regression and Naïve Bayes were (0.89, 0.86, 0.86), (0.86, 0.83, 0.80) and (0.85, 0.81, 0.76), respectively.

Impact and Reach


Activity Overview

Additional statistics for this dataset are available via IRStats2.


Actions (login required)

View Item View Item