Comparing Constraint Evaluation Methods for Shape-Constrained Regression

Shape-constrained regression is an important desideratum of data-based modeling when you want to enforce your model to possess an expected behavior despite the intrinsic noise of the collected data. Conventional data-based modeling approaches are solely reliant on empirical data, and thus, often fail to incorporate the essential physical constraints inherent to the underlying systems. This leads to a lack of trust in the model and a reduction of explanatory power. Recognizing these limitations of traditional data-based methodologies, the integration of shape constraints, derived from domain expertise or fundamental physical principles, emerges as a promising extension to enhance the quality of models. In this paper, we distinguish between single-objective and multi-objective approaches, as well as soft and hard constraint methods to evaluate the constraint violations of models. In the context of this research, the primary focus is on single-objective approaches that incorporate an expanded scope of features and dynamics for constraint handling. Instead of strictly enforcing hard constraints and discarding solutions solely based on constraint violation, the paper presents a dynamic method for calculating constraint violation and prediction error. The presented approach involves an incremental increase in the weighting of constraint violation during the modeling process, achieving an initial, wide-ranging exploration of the solution space followed by a comprehensive investigation of potential models through an in-depth search. Furthermore, we want to explore the critical issue of premature stagnation, a phenomenon wherein the search for a conformant model, plateaus before reaching an acceptable local optima solution. This paper tries to address this challenge through different strategies to maintain a certain diversity of solutions.