TY - CONF TI - Novelty and Diversity in Information Retrieval Evaluation AU - Clarke, Charles L.A. AU - Kolla, Maheedhar AU - Cormack, Gordon V. AU - Vechtomova, Olga AU - Ashkan, Azin AU - Büttcher, Stefan AU - MacKinnon, Ian T3 - SIGIR '08 AB - Evaluation measures act as objective functions to be optimized by information retrieval systems. Such objective functions must accurately reflect user requirements, particularly when tuning IR systems and learning ranking functions. Ambiguity in queries and redundancy in retrieved documents are poorly reflected by current evaluation measures. In this paper, we present a framework for evaluation that systematically rewards novelty and diversity. We develop this framework into a specific evaluation measure, based on cumulative gain. We demonstrate the feasibility of our approach using a test collection based on the TREC question answering track. C1 - New York, NY, USA C3 - Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval DA - 2008/// PY - 2008 DO - 10.1145/1390334.1390446 DP - ACM Digital Library SP - 659 EP - 666 LA - en PB - ACM SN - 978-1-60558-164-4 UR - http://doi.acm.org/10.1145/1390334.1390446 Y2 - 2019/01/27/19:15:02 ER -