Abstract
Large Language Models (LLMs) have become relevant for enterprise support systems,offering new possibilities for intelligent information retrieval, troubleshooting assistance,
and automated access to operational data. However, the integration of such systems
into enterprise environments introduces challenges related to data sovereignty, system
interoperability, maintainability, and reliable access to domain-specific knowledge.
This thesis presents the design, implementation, and evaluation of a fully self-hosted
and modular AI-powered support chatbot for the EMIL charging infrastructure man
agement platform developed by Fronius. The proposed architecture combines retrieval
augmented generation (RAG) and Model Context Protocol (MCP) based tool integra
tion to enable both knowledge-grounded responses and access to live operational data.
During the development process, two architectural approaches were implemented and
evaluated: an Open WebUI-based prototype utilizing a custom Python pipeline and a
custom implementation based on the Vercel AI SDK.
To evaluate the effectiveness of the developed system, a comprehensive evaluation
strategy was conducted consisting of a human-centered user evaluation and an auto
mated LLM-as-a-Judge evaluation using RAGAS-based metrics. The results indicate
that both approaches are capable of providing contextually relevant technical support,
while the AI-SDK-based architecture demonstrated improved response groundedness,
faithfulness, and integration of live operational data through MCP tools. At the same
time, the evaluation revealed persistent challenges regarding hallucinations, contextual
reasoning, and conversational consistency when knowledge is incomplete or unavailable.
The findings demonstrate that self-hosted, tool-augmented chatbot systems can pro
vide valuable support capabilities in enterprise environments while maintaining opera
tional control and data sovereignty. Furthermore, the proposed architecture establishes
a flexible foundation for future enterprise AI assistants and autonomous agent-based
systems.
| Date of Award | 2026 |
|---|---|
| Original language | English (American) |
| Awarding Institution |
|
| Supervisor | Kathrin Maria Preiner (Supervisor) |
Studyprogram
- Mobile Computing
Cite this
- Standard