Skip to main navigation Skip to search Skip to main content

Effects of Obfuscation and Dynamic Debugging Contexts on Bug Fixing with Large Language Models

  • Christoph Daxerer

    Student thesis: Master's Thesis

    Abstract

    Large language models are increasingly used for software development tasks, including
    bug localization and repair. Their reported performance, however, can be difficult to
    interpret when benchmark programs are familiar, clearly written, or rich in surface
    cues such as meaningful identifiers and conventional structure. This thesis investigates
    whether large language models can still identify and repair injected bugs when the
    same programs are transformed through source-code obfuscation, and whether access
    to dynamic debugging tools can compensate for the resulting loss of readability.
    The thesis therefore introduces Grenzer, an experimental framework for constructing
    controlled Python debugging tasks. The thesis uses the framework with a selection of
    executable LeetCode-style programs to inject one observable defect per sample, validate
    the defect with tests, filter for samples that are solvable before obfuscation, applying
    source-level obfuscations while tracking bug locations, and evaluating model diagnoses
    using a panel of judge models. The evaluated bug families are off-by-one errors, wrong
    variables in expressions, wrong logic operators, and wrong arithmetic operators.
    The results show that the evaluated models are robust to mild obfuscation, but not
    invariant to it. Individual transformations cause moderate degradation, with misleading
    comments having the smallest effect and identifier renaming producing clearer drops.
    The strongest static effect appears under combined obfuscation: gpt-5.4 remains at
    91.3% approved repair attempts, while Qwen 3.6 falls to 78.0%. Dynamic debugging
    changes this picture. In the agentic multistep condition, Qwen 3.6 improves to 94.8%
    approval by using probes and debugger actions extensively, whereas gpt-5.4 changes
    only slightly, from 91.3% to 91.7%, rarely using debugger-specific tools. This improve
    ment comes with a substantial token-cost trade-off.
    Overall, the thesis shows that obfuscation is a useful stress test for code-debugging
    ability. Agentic debugging demonstrates that performance can be recovered when models
    are allowed to gather runtime evidence.
    Date of Award2026
    Original languageEnglish (American)
    Awarding Institution
    • FH OÖ
    SupervisorJosef Pichler (Supervisor) & Mario Winterer (Supervisor)

    Studyprogram

    • Software Engineering

    Cite this

    '