Planung und Implementierung eines Retrieval Augmented Generation-Systems im Unternehmenskontext

Abstract

This master’s thesis explores the design and implementation of a Retrieval-Augmented Generation (RAG) system in an organizational context, aiming to enhance information retrieval and usability through a prototype. Using Microsoft Azure and Copilot Studio as services since the organization is already settled with this hybrid infrastructure. Organizational data, such as JIRA tickets and internal directories, was processed and indexed to support a chatbot interface, demonstrating the system’s ability
to deliver precise and contextually relevant answers without extensive LLM fine-tuning.
The RAGAS framework was employed to evaluate the system's performance across
metrics like faithfulness, context relevance, and factual correctness, showing significant
improvements through advanced search and re-ranking techniques. Challenges such as
network security, data permissions, and maintaining data freshness were addressed,
underscoring their importance for successful integration.
The thesis concludes with recommendations for gradual implementation and highlights opportunities for expanding RAG systems for good user experience. This research
provides a solid foundation for future projects and demonstrates the potential of AI
technologies to improve organizational knowledge accessibility.

Date of Award	2025
Original language	German (Austria)
Supervisor	Andreas Stöckl (Supervisor)