| title | layout | description | bodyClass |
|---|---|---|---|
Accepted Papers |
page |
Accepted Papers |
page-about |
- _One-Shot Labeling for Automatic Relevance Estimation{:target="blank" rel="noopener"}
- Sean MacAvaney and Luca Soldaini
- _Evaluating Cross-modal Generative Models Using Retrieval Task{:target="blank" rel="noopener"}
- Shivangi Bithel and Srikanta Bedathur
- _A Comparison of Methods for Evaluating Generative IR{:target="blank" rel="noopener"}
- Negar Arabzadeh and Charles L. A. Clarke
- A Novel Evaluation Framework for Image2Text Generation
- Jia-Hong Huang, Hongyi Zhu, Yixian Shen, Stevan Rudinac, Alessio M. Pacces and Evangelos Kanoulas
- Using LLMs to Investigate Correlations of Conversational Follow-up Queries with User Satisfaction
- Hyunwoo Kim, Yoonseo Choi, Taehyun Yang, Honggu Lee, Chaneon Park, Yongju Lee, Jin Young Kim and Juho Kim
- EXAM++: LLM-based Answerability Metrics for IR Evaluation
- Naghmeh Farzi and Laura Dietz
- _Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems{:target="blank" rel="noopener"}
- Clemencia Siro, Mohammad Aliannejadi and Maarten de Rijke
- _On the Evaluation of Machine-Generated Reports{:target="blank" rel="noopener"}
- James Mayfield, Eugene Yang, Dawn Lawrie, Sean MacAvaney, Paul McNamee, Douglas W. Oard, Luca Soldaini, Ian Soboroff, Orion Weller, Efsun Kayi, Kate Sanders, Marc Mason and Noah Hibbler
- Toward Automatic Relevance Judgment using Vision–Language Models for Image–Text Retrieval Evaluation
- Jheng-Hong Yang and Jimmy Lin
- _Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I.{:target="blank" rel="noopener"}
- Harrie Oosterhuis, Rolf Jagerman, Zhen Qin, Xuanhui Wang and Michael Bendersky
- _Selective Fine-tuning on LLM-labeled Data May Reduce Reliance on Human Annotation: A Case Study Using Schedule-of-Event Table Detection{:target="blank" rel="noopener"}
- Bhawesh Kumar, Jonathan Amar, Eric Yang, Nan Li and Yugang Jia
- _FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions{:target="blank" rel="noopener"}
- Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie and Luca Soldaini
- _The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches{:target="blank" rel="noopener"}
- Bhashithe Abeysinghe and Ruhan Circi
- _Can We Use Large Language Models to Fill Relevance Judgment Holes?{:target="blank" rel="noopener"}
- Zahra Abbasiantaeb, Chuan Meng, Leif Azzopardi and Mohammad Aliannejadi
- _Query Performance Prediction using Relevance Judgments Generated by Large Language Model{:target="blank" rel="noopener"}
- Chuan Meng, Negar Arabzadeh, Arian Askari, Mohammad Aliannejadi and Maarten de Rijke
- _Large Language Models for Relevance Judgment in Product Search{:target="blank" rel="noopener"}
- Navid Mehrdad, Hrushikesh Mohapatra, Mossaab Bagdouri, Prijith Chandran, Alessandro Magnani, Xunfan Cai, Ajit Puthenputhussery, Sachin Yadav, Tony Lee, Chengxiang Zhai and Ciya Liao
- _Evaluating the Retrieval Component in LLM-Based Question Answering Systems{:target="blank" rel="noopener"}
- Ashkan Alinejad, Krtin Kumar and Ali Vahdat
- _Exploring Large Language Models for Relevance Judgments in Tetun{:target="blank" rel="noopener"}
- Gabriel de Jesus and Sérgio Nunes
- A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation
- Weijia Zhang, Mohammad Aliannejadi, Jiahuan Pei, Yifei Yuan, Jia-Hong Huang and Evangelos Kanoulas
- _Evaluating RAG-Fusion with RAGElo: an Automated Elo-based Framework{:target="blank" rel="noopener"}
- Zackary Rackauckas, Arthur Câmara and Jakub Zavrel
- Enhancing Demographic Diversity in Test Collections Using LLMs
- Marwah Alaofi, Nicola Ferro, Paul Thomas, Falk Scholer and Mark Sanderson
- GPT-4 Relevance Labelling can be Fooled by Query Keyword Stuffing
- Marwah Alaofi, Paul Thomas, Falk Scholer and Mark Sanderson