Evaluating Zero-Shot Sim-to-Real Transfer for Defect Segmentation in Pulsed Thermography

Abstract

Deep learning models for automated defect segmentation in active thermography present an innovative idea to improve the speed and reliability of non-destructive testing (NDT) procedures. However in practice, the deployment of such models is often limited by the amount of labeled training data available. This is particularly prominent in safety-critical applications, where the occurrence of defective specimens is minimized during manufacturing processes. In this thesis, the feasibility of training segmentation models exclusively on physics-based simulation data is investigated. Following training, the models are directly evaluated on real measurement data without any fine-tuning, i. e. performing a zero-shot sim-to-real transfer. The training data is generated using finite element simulations of pulsed thermography experiments, with domain randomization techniques applied to reduce the influence of the domain gap. The parameters of the simulation model are systematically varied using Latin Hypercube Sampling to cover a larger range of possible experimental configurations. A novel physics-informed feature engineering technique based on non-uniform temporal sampling is used to select informative time steps from the thermal image sequences as the input for the segmentation model. The model architecture selected in this work is a UNet2D convolutional neural network, which is an established encoder-decoder architecture for image segmentation tasks. In addition, this architecture has also been successfully applied to thermographic data in previous research. For evaluation, first a systematic hyperparameter study is conducted on a synthetic test set to identify the best-performing model configurations. The investigated hyperparameters include temporal resolution, interpolation method for temporal resampling, and different loss functions. The two best-performing models are then selected for evaluation on real measurement data. The evaluation demonstrates that models trained purely on simulation data are able to effectively perform zero-shot sim-to-real transfer for defect types that are well represented in the training distribution. The achieved Recall values are exceeding 95% on synthetic data and above 90% for well-represented materials in real measurements. This work validates simulation-based training as a viable approach for thermographic defect segmentation when training data covers the target defect characteristics.

Date of Award	2025
Original language	English
Supervisor	Gerald Zauner (Supervisor)