E. Kouloumpris and I. Vlahavas. "Markowitz random forest: Weighting classification and regression trees with modern portfolio theory." Neurocomputing, vol. 620, Article 129191, 2024.
Tree-based ensembles such as random forest (RF) are essential methods for supervised learning. Whereas traditional RFs assign equal weights to their trees, significant evidence suggests that tree weighting schemes can enhance predictive performance. Previous works have focused solely on the predictive performance of individual trees, assigning greater weights to high-performing trees. However, the predictive power of RF arises not only from high performing trees but also from tree variety, a factor that has not been considered before. In this paper we propose Markowitz RF, a tree weighting method that considers both tree performance and variety, using a tree covariance matrix for risk regularization. Our method is formulated as an adapted optimization process inspired by financial mathematics. It can be applied to binary and multi-class classification, as well as regression tasks. Our experiments on 15 benchmark datasets indicate that MRF can significantly outperform previously proposed tree weighting approaches and other learning methods in terms of Precision-Recall AUC, mean absolute error and F1_macro.
