Can AI replace humans? Comparing the capabilities of AI tools and human performance in a business management education scenario

Herath, Dinuka B ORCID: https://orcid.org/0000-0001-9235-7004, Ode, Egena and Herath, Gayanga B ORCID: https://orcid.org/0000-0002-4014-8339 (2025) Can AI replace humans? Comparing the capabilities of AI tools and human performance in a business management education scenario. British Educational Research Journal. ISSN 0141-1926

Preview

Published Version Available under License Creative Commons Attribution. Download (3MB) | Preview

Abstract

This study provides a comparative assessment of the capabilities of leading artificial intelligence (AI) tools and human participants in a business management education context. Specifically, we (a) assess how well current language models perform in providing answers to standardised essay‐type assessments in a business and management education context, (b) examine the efficacy of emergent tools in detecting AI‐generated texts and (c) evaluate online AI rewriting and paraphrasing tools and their efficacy in evading detection. Using an exploratory qualitative design, this study generated and evaluated 15 standard essays using ChatGPT (n = 5), Bard (n = 5) and human (n = 5). A comparison is provided between the average performance of AI‐derived essays and that of ChatGPT‐generated essays across all five essays. The results suggest that AI‐generated content can achieve reasonably high marks in management and business assessments. According to the findings of the study, AI's performance is highly influenced by the types of prompts used, the user's experience and the degree to which the user can discern between relevant and irrelevant content. According to the findings, Turnitin's AI detection tool is highly effective at detecting content that has been created by AI, but the effectiveness is reduced by rewriters. The Turnitin AI detection tool, however, is significantly more effective at identifying content generated by Bard compared with content generated by ChatGPT. According to the results, ChatGPT produced better results when the user provided a clear context, outlined the topic and expectations, divided the assessment tasks into sections and fed the prompts in a conversational manner to train the model. By utilising AI chatbots effectively, traditional teaching and assessment methods can be supplemented with targeted and engaging learning experiences.