Resource type

A systematic review of interactive information retrieval evaluation studies, 1967–2006

Resource type
Authors/contributors
Title
A systematic review of interactive information retrieval evaluation studies, 1967–2006
Abstract
With the increasing number and diversity of search tools available, interest in the evaluation of search systems, particularly from a user perspective, has grown among researchers. More researchers are designing and evaluating interactive information retrieval (IIR) systems and beginning to innovate in evaluation methods. Maturation of a research specialty relies on the ability to replicate research, provide standards for measurement and analysis, and understand past endeavors. This article presents a historical overview of 40 years of IIR evaluation studies using the method of systematic review. A total of 2,791 journal and conference units were manually examined and 127 articles were selected for analysis in this study, based on predefined inclusion and exclusion criteria. These articles were systematically coded using features such as author, publication date, sources and references, and properties of the research method used in the articles, such as number of subjects, tasks, corpora, and measures. Results include data describing the growth of IIR studies over time, the most frequently occurring and cited authors and sources, and the most common types of corpora and measures used. An additional product of this research is a bibliography of IIR evaluation research that can be used by students, teachers, and those new to the area. To the authors' knowledge, this is the first historical, systematic characterization of the IIR evaluation literature, including the documentation of methods and measures used by researchers in this specialty.
Publication
Journal of the American Society for Information Science and Technology
Volume
64
Issue
4
Pages
745-770
Date
April 1, 2013
Journal Abbr
J Am Soc Inf Sci Tec
Language
en
DOI
10.1002/asi.22799
ISSN
1532-2890
Citation
Kelly, D., & Sugimoto, C. R. (2013). A systematic review of interactive information retrieval evaluation studies, 1967–2006. Journal of the American Society for Information Science and Technology, 64(4), 745–770. https://doi.org/10.1002/asi.22799
Field of study