Comparative Analysis of Model Selection Criteria for Symbolic Regression Using Genetic Programming

Symbolic regression (SR) using genetic programming (GP) can generate a diverse set of candidate models that balance accuracy and complexity, particularly when configured with multi-objective optimisation, which produces a Pareto front of non-dominated solutions. However, selecting a single model from this population remains challenging, especially when relying only on training data. This study evaluates the effectiveness of model selection criteria in SR, which include Mean Squared Error (MSE), Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Description Length (DL), and PySR Score Metric (PSM). These criteria are evaluated using their training scores on 20 real-world regression datasets from the PMLB collection using PySR. We calculate the Spearman rank correlation coefficient (ρ) between each metric and test MSE to evaluate how well the metric ranks generalisable models. The results show that no single metric performs reliably across all datasets. Metrics that focus mainly on accuracy often lead to overfitting, while simplicity-based metrics can underfit. PSM aims to balance accuracy and complexity, but its performance is inconsistent, sometimes helpful, but often unstable across datasets. This study provides practical insights into the behaviour of model selection metrics in SR and offers guidance for selecting models that generalise well without overfitting.