Application of LSTM and Transformer Hybrid Model for Electricity Consumption Forecasting
Yaling Zhang *
School of Mathematics and Physics, Southwest University of Science and Technology, Mianyang, 621010, China.
*Author to whom correspondence should be addressed.
Abstract
With the growing energy demand and the increasing intelligence of power systems, accurate electricity load forecasting is crucial for grid scheduling and energy management. In recent years, deep learning methods have made significant progress in time series forecasting, with Transformer and LSTM structures gaining attention for their powerful sequence modeling capabilities. Therefore, this study proposes three different combinations of LSTM and Transformer: (1) Parallel Fusion Model ; (2) LSTM - Transformer; and (3) Transformer -LSTM. Additionally, the Particle Swarm Optimization (PSO) algorithm is employed to optimize the model hyperparameters. The experimental dataset consists of hourly electricity consumption data from Romania, and multi-step forecasting is performed based on the Nonlinear AutoRegressive with Exogenous inputs (NARX) model. The predictive performance of these three hybrid models is compared with traditional machine learning models. The experimental results show that different combinations exhibit varying performances in short-term, mid-term, and long-term forecasting tasks. Specifically, the LSTM-Transformer model performs better in midterm forecasting, while the Transformer-LSTM model excels in long-term forecasting.Since this study compares hybrid models for multi-step electricity load forecasting in a specific country, it can reveal the effectiveness at different time scales. This provides new insights for electricity load forecasting and offers valuable references for subsequent research and practical applications.
Keywords: Transformer, LSTM, particle swarm optimization, electricity load forecasting