Transformer-based low-complexity time series forecasting model

• 1、School of Computer Science,Beijing University of Posts and Telecommunications, Beijing 100083

Abstract：Time series forecasting is to predict future data based on historical time series data, which has an important role in retail, finance, power system and other fields, and it is of great significance to make accurate time series forecasting. Existing deep learning models such as RNN and LSTM have achieved many results in the field of time series forecasting, but these models generally have the disadvantage of poor accuracy of long time series prediction because of their structural features that it is difficult to capture the long-dependent features of time series data. transformer adopts the self-attentive mechanism, which has the characteristics of parallel processing and sensitive to long-dependent features, but Transformer limits its application space on long time series forecastingproblems because of its quadratic complexity problem. To address the quadratic complexity problem of Transformer, we propose the Max Self-attention mechanism, which has a lower time complexity than Self-attention, is faster and has higher prediction accuracy in long time seriesforecasting problems.

Keywords： time series forecasting self-attentive mechanisms numerical prediction deep learning