Enhancing User Interfaces in High-Latency RAG Systems A Study on UI Design and Usability

Abstract

Recent advances in artificial intelligence have significantly improved chatbot interactions, primarily through the integration of large language models which are capable of generating coherent responses. Despite these advancements, purely generative chatbots often encounter issues such as inaccuracies or hallucinations. Retrieval-Augmented Generation (RAG) addresses these limitations by integrating external knowledge sources to retrieve information directly from them. This increases accuracy and reliability. However, implementing RAG introduces new user interface and usability challenges. Therefore, this thesis investigates UI design solutions to optimize interactions within RAG systems, focusing specifically on how citations are displayed and confidence scores are presented. Furthermore, the integration of compute intensive reasoning models, such as OpenAI’s o1 model, leads to a longer waiting time and makes latency management a central challenge in UI design. A prototype chatbot was developed to evaluate different design components through a user study, combining qualitative and quantitative methods. Based on the insights of the study, comprehensive design guidelines are formulated that reveal effective strategies for citation visibility, intuitive confidence scoring, and engaging feedback during high-latency scenarios. This work contributes practical recommendations that will support developers in designing more reliable, intuitive, and user-centered RAG-based chatbots.

Date of Award	2025
Original language	English
Supervisor	Andreas Stöckl (Supervisor)