Application of 2D human pose estimation for automated hold suggestion for visually impaired climbers

Abstract

This thesis investigates the application of 2D human pose estimation models in the
context of climbing, with the primary goal of leveraging computer vision to enable
visually impaired climbers to train more independently.
A novel dataset consisting of 23 videos featuring 11 different climbers ascending two
routes is introduced. The dataset includes information about the pixel locations of holds
in the video, the holds used by the climber in each video, and a homography matrix to
calculate real-world distances.
An initial evaluation of ten 2D human pose estimation models was conducted by
overlaying their detections on the video to assess their effectiveness in identifying climbing poses. The three most accurate and reliable models, ViTPose L, YOLOv8-pose X,
and MediaPipe H, are then used in combination with climbing hold positions to identify
the holds currently used by the climber. The performance of the models for obtaining
these predictions is thoroughly investigated on the custom dataset, with ViTPose L
demonstrating the highest accuracy, albeit at the cost of taking twice as long to predict
a single pose compared to the other two models.
Additionally, two approaches for predicting the next hold are discussed: one rulebased and the other utilizing a database.
Future work will focus on refining the prediction models and expanding the dataset
to include more diverse climbing scenarios, further improving the system’s applicability
in real-world settings.

Date of Award	2024
Original language	English (American)
Supervisor	David Christian Schedl (Supervisor)