Using the information seeker to elicit construct models for search engine evaluation

Crudge, Sarah E. and Johnson, Frances C. (2004) Using the information seeker to elicit construct models for search engine evaluation. Journal of American Society of Information Science and Technology, 55 (9). pp. 794-806. ISSN 1532-2882

Preview

Available under License In Copyright. Download (173kB) | Preview

Abstract

Users’ internal representations of their interactions with systems are often termed ‘mental models’, and for successful system use, the users’ mental models and system designers’ conceptual models of the tools should be congruent. This study explores a method for non-biased determination of the user’s subconscious view of Internet search engines, in order to derive a mental model comprising those aspects of the systems of importance to the users. The investigation utilises a repertory grid approach in combination with laddering technique, the latter being based on the cause and effect style of mental model development. The detailed qualitative analysis of the data determined through use of laddering interviews is presented here in the development of a mental model comprising three strata. The main hierarchical stratum of the model conveys the interrelations between basic system description, evaluative description, and the key evaluations of ease, efficiency, effort and effectiveness. Two additional strata relating to the perceived process and the experience of emotion are also discussed. The conjunction of the procedural elements with the key evaluations is of particular significance, and further research proposes the extension of this to provide a framework for search engine evaluation.