Incorporation of structural properties of the response surface into oblique model trees

Oblique model trees are nonparametric regression models that partition the input space using axis-oblique splits to define local regions for regression. The Hierarchical Local Model Tree algorithm (HILOMOT) efficiently constructs these trees and produces a continuous output from overlapping local models, making it well suited for industrial applications. We enhance HILOMOT with a novel semi-greedy splitting method to reduce the short-sightedness of its greedy splitting, eliminate an undesirable trade-off during tree construction that can cause underfitting, and improve split interpretability. Our approach first estimates the strongest direction of curvature in a partition that the existing model cannot approximate. Then, a non-greedy method determines the optimal split orthogonal to this direction. We compare our method in experiments on synthetic data against HILOMOT and other established regression techniques. Qualitative results confirm that our method produces more interpretable splits, while quantitative results show it generally constructs more accurate and compact trees. Regardless of the splitting method, HILOMOT consistently outperforms all reference methods in accuracy.