e-space
Manchester Metropolitan University's Research Repository

    Task dimensions of user evaluation of information retrieval systems

    Johnson, Frances C., Griffiths, Jillian R. and Hartley, Richard J. (2003) Task dimensions of user evaluation of information retrieval systems. Information Research, 8 (4). ISSN 1368-1613

    [img]
    Download (76kB)

    Abstract

    This paper reports on the evaluation of three search engines using a variety of user-centred evaluation measures grouped into four criteria of retrieval system performance. This exploratory study of users' evaluations of search engines took as its premise that user system success indicators will derive from the retrieval task the system supports (in its objective to facilitate search). This resulted in the definition of user evaluation as a multidimensional construct which provides a framework to link evaluations to system features in defined user contexts. Our findings indicate that users' evaluations across the engines will vary, and the dimensional approach to evaluation suggests the possible impact of system features. Further analysis suggests a moderating effect on the strength of the evaluation by a characterization of the user and/or query context. The development of this approach to user evaluation may contribute towards a better understanding of system feature and contextual impact on user evaluations of retrieval systems.

    Impact and Reach

    Statistics

    Activity Overview
    6 month trend
    381Downloads
    6 month trend
    362Hits

    Additional statistics for this dataset are available via IRStats2.

    Actions (login required)

    View Item View Item