Execution Time Optimization Through Feature and Temporal Reduction in Asset Pricing

Abstract

High-dimensional financial machine learning (ML) pipelines are computational workloads as much as predictive models: their practical value depends on runtime, memory footprint, scalability, and the ability to retrain under resource constraints. This paper treats empirical asset pricing as a demanding real-world workload and proposes Cost-Aware Adaptive Optimization, using XGBoost (extreme gradient boosting), as a systems-aware framework to improve the computational efficiency of gradient-boosted tree pipelines while preserving application-level predictive utility. The framework uses computational feedback; optimization is driven by training time, prediction time, memory footprint, feature dimensionality, temporal training-window depth, model complexity, and marginal utility per unit of computation. We first profile 10 ML algorithms in a controlled high-performance computing setting and decompose execution cost into training and prediction components. XGBoost is then selected as the optimization target because it offers a strong cost–utility baseline on the asset-pricing workload. The proposed framework adaptively controls feature-dimensionality and temporal-depth choices, evaluates memory- and execution-aware trade-offs, and analyzes scalability through runtime growth, parallel speed-up, and parallel efficiency. Application-level utility measures, including out-of-sample R2, mean squared error (MSE), and the Sharpe ratio of decile-sorted long–short portfolios, are retained as application-level validation metrics rather than as the sole objective. The results show that meaningful computational savings can be obtained by treating dimensionality, temporal depth, and model complexity as controllable workload variables. The study therefore contributes a computational feedback-driven optimization framework for scalable financial ML execution, framing workload adaptation and configuration selection as runtime-, memory-, and utility-aware system problems. © 2026 The Author(s). Concurrency and Computation: Practice and Experience published by John Wiley & Sons Ltd.