Keypoint Detection on Images of Burn Injuries

Abstract

In this thesis, the foundation for semi-automated mapping of burn wounds from 2D images to a 3D avatar based on anatomical landmarks is elaborated. For this purpose, suitable datasets and human pose estimation are applied and evaluated. As the dataset, the MPII images with TRB annotations are employed. The TRB annotations not only contain the usual body joint points, but also marks corresponding contour points. The Simple Baseline Model and Stacked Hourglass architecture are utilized for human pose estimation. The model implementation is based on the MMPose Framework and adapted according to the requirements of this thesis. RISC Software GmbH provided a small internal dataset with approximately 100 images of burn patients, from which 89 images were finally annotated accordingly. Due to the nature of the injuries, patients are often photographed from a non-standardized perspective. As in most cases, only the affected area is depicted, while larger parts of the body are not shown. Another aspect that distinguishes the images from typical datasets is the depiction of people in a hospital environment. Some of them are unclothed and disfigured by the wounds. This is the fundamental challenge in this work. As no other suitable burns datasets were found during the research, the burn images can only be used as a test dataset due to their small number. Nevertheless, in order to optimize the selected architectures for the challenges of wound images, image sections were generated specifically according to body regions from the MPII-TRB images. Both architectures are trained and evaluated on the subsets of the provided annotations: the body, contour and TRB annotations of the data. Each of these model instances performed well on the MPII-TRB images, regardless of whether they were evaluated on the original or cropped images. However, the accuracy of all instances drops when applied to wound images. Fine-tuning the models again using the generated cutouts cannot improve this. Therefore, the differences between wound images and MPII images should be considered in the future, possibly by adding images from the hospital setting. In addition, the complexity of the model architectures used, as well as the size of the input images and the heat maps to be generated to determine the keypoints, can be increased to improve the generalizability of the architectures.

Date of Award	2025
Original language	English
Supervisor	David Christian Schedl (Supervisor) & Sophie Kaltenleithner (Supervisor)