Enhancing Photovoltaic Plant Maintenance Efficiency through Augmented Reality

Abstract

Locating specific photovoltaic (PV) modules in large plants can be a time-consuming and error-prone task. This thesis investigates how Augmented Reality (AR) and object detection can be used on mobile devices to enable technicians to find, allocate and verify modules in real-time on site. The developed Unity prototype retrieves and shows live PV plant performance data and uses YOLOv8, combined with an approach that calculates the expected position of each PV module in the PV plant, to examine whether this concept can enhance the maintenance efficiency through AR. Hereby, the system utilizes a Vuforia image target, placed on the surface of a first starting module, so that it provides pivotal information about the position and orientation of the module’s surface as an initial reference point. From this anchor point, expected positions of all modules are calculated based on an approach that converts each module’s position into a plant coordinate format. After running inference using a YOLOv8 model, the system then uses the calculated distances to allocate the detected PV module bounding boxes to their corresponding real-world counterpart, by applying a best match algorithm. Virtual labels are then instantiated over the respective modules, while the system re-anchors to the nearest detected module. Two YOLOv8 variants were trained, whereas the nano model achieved a higher Precision for PV modules (0,7578 vs. 0,7211) and a lower end-to-end latency (4,47s vs 5,39s), despite being smaller in size compared to a small model. Field validation on two rooftop PV plants showed that stricter confidence and Intersection over Union (IoU) thresholds yielded more reliable allocation, with approximately three quarters of all modules correctly verified. Even though the prototype was vulnerable to reflections, short distances and poor anchor placement, the results reveal that combining AR with object detection is feasible on mobile devices and can support the PV maintenance for small to medium plants, while providing a basis for further improvements and scalability on large-scale plants.

Date of Award	2025
Original language	English
Supervisor	Jens Krösche (Supervisor)