Echtzeit-Erkennung illegaler Handgesten unter Verwendung von Convolutional Neural Networks

Abstract

Object recognition is an artificial intelligence method used to automatically identify and classify relevant objects in image and video data. This master's thesis focuses on the real-time detection of legally prohibited hand gestures in Austria, specifically the nazi salute (also known as the „hitler salute”) and the wolf salute, using convolutional neural networks.
The aim is to create an appropriate dataset and to investigate the technical feasibility and reliability of an object detection model for these gestures. A central challenge is data acquisition, as there is no publicly annotated dataset available for these gestures. To ad-dress this problem, the challenges of data acquisition are analyzed, as they have a significant impact on the reliability of the object detection model. A class-balanced dataset with 800 training images and 1719 annotated object instances was created. The dataset exhibits high intra-class variation, for example, due to different perspectives, scales, and back-grounds, which ensures realistic coverage of potential use cases. Furthermore, the influence of dataset size on model performance is investigated, and transfer learning is incorporated into the experiments. The YOLO (You Only Look Once) model, which is considered the current state of the art, is used for object detection. The evaluation is based on established metrics on validation and test data. The best performance was achieved by the pretrained model trained on 800 images, with a mean average precision of 90% at a threshold of 0.5. The results clearly demonstrate that a balanced and diverse dataset, combined with transfer learning, contributes significantly to the model's performance accuracy.
This work not only demonstrates the technical feasibility of real-time detection of illegal hand gestures but also provides a systematically prepared data basis for future research or applications in the field of public security and content moderation on social media. At the same time, the remaining misclassifications indicate that further optimizations regarding dataset size, annotation, hyperparameter tuning or model architecture adjustments are necessary to further increase robustness and accuracy.

Date of Award	2025
Original language	German (Austria)
Supervisor	Oliver Krauss (Supervisor)