Enhanced very short-term load forecasting with multi-lag feature engineering and prophet-XGBoost-CatBoost architecture

Shafiuzzaman, Md ORCID: https://orcid.org/0009-0001-5217-6070, Safayet Islam, Md, Rubaith Bashar, T.M, Munem, Mohammad, Nahiduzzaman, Md, Ahsan, Mominul and Haider, Julfikar ORCID: https://orcid.org/0000-0001-7010-8285 (2025) Enhanced very short-term load forecasting with multi-lag feature engineering and prophet-XGBoost-CatBoost architecture. Energy, 335. 137981. ISSN 0360-5442

Preview

Published Version Available under License Creative Commons Attribution. Download (13MB) | Preview

Abstract

An efficient very short-term load forecast (VSTLF) is essential for power plant unit scheduling because it reduces the planning uncertainty caused by variable renewable energy outputs, which in turn decreases the overall electricity production costs in a power production system. Even though this sector has seen a number of studies and applications in plant scheduling, improving projections with minimum errors is still necessary. In this paper, a novel machine learning framework for predicting very short-term electricity usage was developed. The data utilized in this study are publicly available and were accessed from Kaggle's platform. The framework involved two pivotal stages in its development: robust feature engineering and electric load forecasting via a Prophet-XGBoost-CatBoost (Pr-XGB-CB) model. Feature engineering involves the use of multiple sets of historical electricity consumption data with different time lags for analysis and enhancement. Prophet dissects forecasted data into understandable seasonal, trend, and holiday segments, whereas XGBoost stands out for its speed and effectiveness, especially when dealing with numerous features. To obtain the ensembled forecast, the CatBoost algorithm was utilized. The optimal hyperparameters for the model are evaluated by Optuna. The effectiveness of the suggested model was evaluated by contrasting its performance with that of five alternative machine learning (ML) and deep learning (DL) algorithms. In addition, a sensitivity analysis was conducted to assess the model's robustness under scenarios of missing or limited data, demonstrating its resilience in real-world applications. The proposed framework outperformed the state-of-the-art (SOTA) models, with a Mean Absolute Error (MAE) of 23.70, Root Mean Squared Error (RMSE) of 32.32, and an R² of 0.97; hence, the proposed framework performs an efficient guiding role in electrical load prediction. This research offered practical significance by enhancing power plant scheduling efficiency and reducing overall electricity production costs through superior predictive accuracy.