Improving job scheduling on a heterogeneous cluster by predicting job execution times using heuristics

In this paper, we propose the scheduling system for the Bioinformatics Resource Facility Hagenberg (BiRFH). This system takes advantage of the fact that the facility offers tailored solutions for the customers, which includes having a limited amount of different programs available. Additionally, the BiRFH system provides access to different hardware platforms (standard CPU, GPGPU on NVIDIA Cuda, and IMB Cell on Sony Playstation machines) with multiple versions of the same algorithm optimized for these platforms. The BiRFH scheduling system takes these into account and uses knowledge about past runs and run times to predict the expected run time of a job. That leads to a better scheduling and resource usage. The prediction and scheduling use heuristic and artificial intelligence methods to achieve acceptable results. The paper presents the proposed prediction method as well as an overview of the scheduling algorithm.