Manchester Metropolitan University's Research Repository

    Sentiment analysis of tweets through Altmetrics: a machine learning approach

    Hassan, Saeed-Ul, Saleem, Aneela, Soroya, Saira Hanif, Safder, Iqra, Iqbal, Sehrish, Jamil, Saqib, Bukhari, Faisal, Aljohani, Naif Radi and Nawaz, Raheel ORCID logoORCID: https://orcid.org/0000-0001-9588-0052 (2021) Sentiment analysis of tweets through Altmetrics: a machine learning approach. Journal of Information Science, 47 (6). pp. 712-726. ISSN 0165-5515

    Accepted Version
    Available under License Creative Commons Attribution Non-commercial No Derivatives.

    Download (542kB) | Preview


    The purpose of the study is to (a) contribute to annotating an Altmetrics dataset across five disciplines, (b) undertake sentiment analysis using various machine learning and natural language processing–based algorithms, (c) identify the best-performing model and (d) provide a Python library for sentiment analysis of an Altmetrics dataset. First, the researchers gave a set of guidelines to two human annotators familiar with the task of related tweet annotation of scientific literature. They duly labelled the sentiments, achieving an inter-annotator agreement (IAA) of 0.80 (Cohen’s Kappa). Then, the same experiments were run on two versions of the dataset: one with tweets in English and the other with tweets in 23 languages, including English. Using 6388 tweets about 300 papers indexed in Web of Science, the effectiveness of employed machine learning and natural language processing models was measured by comparing with well-known sentiment analysis models, that is, SentiStrength and Sentiment140, as the baseline. It was proved that Support Vector Machine with uni-gram outperformed all the other classifiers and baseline methods employed, with an accuracy of over 85%, followed by Logistic Regression at 83% accuracy and Naïve Bayes at 80%. The precision, recall and F1 scores for Support Vector Machine, Logistic Regression and Naïve Bayes were (0.89, 0.86, 0.86), (0.86, 0.83, 0.80) and (0.85, 0.81, 0.76), respectively.

    Impact and Reach


    Activity Overview
    6 month trend
    6 month trend

    Additional statistics for this dataset are available via IRStats2.


    Repository staff only

    Edit record Edit record