Skip to main navigation Skip to search Skip to main content

Design and Integration of an LLM-based Diagnostic System for Error Analysis in Energy Infrastructure Devices

  • Florian Spitzbart

    Student thesis: Master's Thesis

    Abstract

    Large Language Models (LLMs) have become relevant for enterprise support systems,
    offering new possibilities for intelligent information retrieval, troubleshooting assistance,
    and automated access to operational data. However, the integration of such systems
    into enterprise environments introduces challenges related to data sovereignty, system
    interoperability, maintainability, and reliable access to domain-specific knowledge.
    This thesis presents the design, implementation, and evaluation of a fully self-hosted
    and modular AI-powered support chatbot for the EMIL charging infrastructure man
    agement platform developed by Fronius. The proposed architecture combines retrieval
    augmented generation (RAG) and Model Context Protocol (MCP) based tool integra
    tion to enable both knowledge-grounded responses and access to live operational data.
    During the development process, two architectural approaches were implemented and
    evaluated: an Open WebUI-based prototype utilizing a custom Python pipeline and a
    custom implementation based on the Vercel AI SDK.
    To evaluate the effectiveness of the developed system, a comprehensive evaluation
    strategy was conducted consisting of a human-centered user evaluation and an auto
    mated LLM-as-a-Judge evaluation using RAGAS-based metrics. The results indicate
    that both approaches are capable of providing contextually relevant technical support,
    while the AI-SDK-based architecture demonstrated improved response groundedness,
    faithfulness, and integration of live operational data through MCP tools. At the same
    time, the evaluation revealed persistent challenges regarding hallucinations, contextual
    reasoning, and conversational consistency when knowledge is incomplete or unavailable.
    The findings demonstrate that self-hosted, tool-augmented chatbot systems can pro
    vide valuable support capabilities in enterprise environments while maintaining opera
    tional control and data sovereignty. Furthermore, the proposed architecture establishes
    a flexible foundation for future enterprise AI assistants and autonomous agent-based
    systems.
    Date of Award2026
    Original languageEnglish (American)
    Awarding Institution
    • FH OÖ
    SupervisorKathrin Maria Preiner (Supervisor)

    Studyprogram

    • Mobile Computing

    Cite this

    '