Evaluating a managed learning environment in a UK higher education institution: a stakeholder approach

Hardman, Julie Ann (2013) Evaluating a managed learning environment in a UK higher education institution: a stakeholder approach. Doctoral thesis (PhD), Manchester Metropolitan University.

Preview

Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (19MB) | Preview

Abstract

Managed Learning Environments (MLE) in Higher Education Institutions (HEIs), as a concept, are relatively new to the arena of Higher Education; nevertheless over 90% of institutions in the Higher and Further Education sector have been engaged in some kind of MLE development activity (University of Brighton, 2005). However, this increased use of learning technology has not produced a concomitant rise in appropriate forms of evaluation (Tricker et al., 2001; Bullock & Ory, 2000). There are no universally recognisable frameworks for evaluating MLEs in HEIs currently discussed within the literature. A review of the literature highlighted the importance of stakeholder involvements in the evaluation process. It was found that an appropriate framework for evaluation needs to be able to: capture the locally situated version of an MLE; cope with the complexity of a system with an unspecified number of variables; identify and encompass stakeholder needs; and understand why certain phenomena has been observed. Soft Systems Methodology (SSM) was considered to be an appropriate framework to cope with all these issues. It uses systems thinking as its theoretical base and one of the major strengths, from the point of view of this research, is its ability to cope explicitly with differing stakeholder views via the concept of Weltanschauung - the ‘world-view’ of different social actors (Rose & Haynes, 1999). This longitudinal study was conducted using a dual-cycle (McKay & Marshall, 2001) Action Research approach. The host university was Manchester Metropolitan University who, at the start of this research project, began a phased implementation of an MLE. An evaluation model (Rose & Haynes, 1999) was used which was adjusted to allow for a stakeholder analysis to drive the evaluation criteria.This study found SSM fulfilled the requirements of evaluation and so was considered a suitable approach. The study did however conclude that by contextualising SSM to the evaluation requirements of an MLE in a UK HEI, the measures of performance suggested by SSM may need to be adjusted. Four out of the five measures of performance were found to adequately provide the evaluation criteria. Ethicality was the only measure of performance found to not be considered as an explicit measure of the information system under study. Identification of stakeholders and encompassing their needs within evaluations were seen as key. This study found that a stakeholder classification framework, offered by Farbey et al. (1993), proved suitable in identifying relevant stakeholders to an MLE. It established that the framework facilitated a holistic representation of the key stakeholders and their views on key metrics on which to evaluate the MLE in situ. This research was also interested in the process of evaluation. The processes utilised were adapted and adjusted over time and a number of key elements are proposed in order to gain efficiencies in resource requirements throughout the evaluation process.