Intelligent Abstractive Summarization of Scholarly Publications with Transfer Learning

Zaman, Farooq, Afzal, Munaza, Teh, Pin Shen ORCID: https://orcid.org/0000-0002-0607-2617, Sarwar, Raheem ORCID: https://orcid.org/0000-0002-0640-807X, Kamiran, Faisal ORCID: https://orcid.org/0000-0002-1168-9451, Aljohani, Naif R. Aljohani ORCID: https://orcid.org/0000-0001-9153-1293, Nawaz, Raheel ORCID: https://orcid.org/0000-0001-9588-0052, Hassan, Muhammad Umair ORCID: https://orcid.org/0000-0001-7607-5154 and Sabah, Fahad (2024) Intelligent Abstractive Summarization of Scholarly Publications with Transfer Learning. Journal of Informatics and Web Engineering, 3 (3). pp. 256-270. ISSN 2821-370X

Preview

Published Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (861kB) | Preview

Abstract

Intelligent abstractive text summarization of scholarly publications refers to machine-generated summaries that capture the essential ideas of an article while maintaining semantic coherence and grammatical accuracy. As information continues to grow at an overwhelming rate, text summarization has emerged as a critical area of research. In the past, summarization of scientific publications predominantly relied on extractive methods. These approaches involve selecting key sentences or phrases directly from the original document to create a summary or generate a suitable title. Although extractive methods preserve the original wording, they often lack the ability to produce a coherent, concise, and fluent summary, especially when dealing with complex or lengthy texts. In contrast, abstractive summarization represents a more sophisticated approach. Rather than extracting content from the source, abstractive models generate summaries using new language, often incorporating words and phrases not found in the original text. This allows for more natural, human-like summaries that better capture the key ideas in a fluid and cohesive manner. This study introduces two advanced models for generating titles from the abstracts of scientific articles. The first model employs a Gated Recurrent Unit (GRU) encoder coupled with a greedy-search decoder, while the second utilizes a Transformer model, known for its capacity to handle long-range dependencies in text. The findings demonstrate that both models outperform the baseline Long Short-Term Memory (LSTM) model in terms of efficiency and fluency. Specifically, the GRU model achieved a ROUGE-1 score of 0.2336, and the Transformer model scored 0.2881, significantly higher than the baseline LSTM model, which reported a ROUGE-1 score of 0.1033. These results underscore the potential of abstractive models to enhance the quality and accuracy of summarization in academic and scholarly contexts, offering more intuitive and meaningful summaries.