A utilisation focussed and viable systems approach for evaluating technology supported learning

Hart, Diane ORCID: https://orcid.org/0000-0003-2642-7814 and Paucar-Caceres, Alberto ORCID: https://orcid.org/0000-0002-4690-561X (2017) A utilisation focussed and viable systems approach for evaluating technology supported learning. European Journal of Operational Research, 259 (2). pp. 626-641. ISSN 0377-2217

Preview

Published Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (1MB) | Preview

Abstract

The paper uses a higher education case study to illustrate a participative theory of change approach to evaluating technology supported learning. The approach is informed by the Viable Systems Model (VSM) and utilisation-focussed evaluation and, falls within the tradition of facilitated modelling approaches to operational research. We argue that this approach worked well in engaging primary evaluation users in a process of collaborative action research to improving an educational development initiative and that the approach helped generate information relevant to answering its primary users’ questions, to inform their specific decisions and actions relevant to their quality enhancement responsibilities. Through a case study, concerning the evaluation of an educational development initiative in a large UK university, we illustrate how the VSM and utilisation-focussed evaluation could be used to: (a) conceptualise the connection between strategies and their components at different levels of organisation; (b) to clarify the role and interests of stakeholders in these strategies; and (c) to scope evaluation to be relevant to informing the decisions and actions of these stakeholders. The paper contributes to illustrate how VSM principles can underpin a theory of change approach to engaging primary stakeholders in planning an intervention and its evaluation in the context of educational development work, in order to improve evaluation to be more relevant to their needs. The paper should be of interest to researchers exploring the use of systems theory in evaluation, in particular in the context of educational development work in higher education.