Spatiotemporal Data Prediction Model Based on a Multi-Layer Attention Mechanism

Spatiotemporal Data Prediction Model Based on a Multi-Layer Attention Mechanism

Man Jiang, Qilong Han, Haitao Zhang, Hexiang Liu
Copyright: © 2023 |Pages: 15
DOI: 10.4018/IJDWM.315822
Article PDF Download
Open access articles are freely available for download

Abstract

Spatiotemporal data prediction is of great significance in the fields of smart cities and smart manufacturing. Current spatiotemporal data prediction models heavily rely on traditional spatial views or single temporal granularity, which suffer from missing knowledge, including dynamic spatial correlations, periodicity, and mutability. This paper addresses these challenges by proposing a multi-layer attention-based predictive model. The key idea of this paper is to use a multi-layer attention mechanism to model the dynamic spatial correlation of different features. Then, multi-granularity historical features are fused to predict future spatiotemporal data. Experiments on real-world data show that the proposed model outperforms six state-of-the-art benchmark methods.
Article Preview
Top

Introduction

With the development of the mobile Internet, the requirements for data processing efficiency and mining depth are increasing rapidly (Zheng, Y., et al., 2014). As an important research field of spatiotemporal data processing, it is of great significance in accelerating the process of smart city construction (Bai, L., et al., 2019) and manufacturing (Ge, Z., et al., 2011) in China and has been widely applied in a number of scenarios, such as air quality prediction (Wang, J., et al., 2021; Pan, Z., et al., 2019), traffic flow prediction (Pan, Z., et al., 2019; Gong, Y., et al., 2019; Pan, Z., et al., 2022), medical risk prediction (Ye, M., et al., 2020) and industrial production prediction (Cho, S., et al., 1997).

Current spatiotemporal data prediction models make some achievements. ARIMA (George, E., & Gwilym, M., 1976) enables the extraction of the linear relationships between data while ignoring the complex nonlinear relationships, leading to low accuracy. ANN (Hopfield, J. J., 1982) represents complex nonlinear functions with an integrated structure of linear threshold units and partly discovers medium- and long-term patterns of spatiotemporal data (Martin, T., et al., 2017). However, the results are sensitive to the initial random weights and thresholds, which would be quite challenging in the case of real industrial production owing to the high demand for reliability. Support vector machines (SVMs) (Bernhard, E., et al., 1992) are designed to map the input vector to a high-dimensional space and analyze the nonlinear characteristics of the sample using a linear algorithm to improve the accuracy (Sapankevych, N., & Sankar, R., 2009). Unfortunately, it is difficult to build a common prediction model because of the prior domain knowledge and the sensitivity of parameters and kernel functions. Random forests (Emmanouil, A., et al., 2019) train multiple decision trees for joint prediction. These methods are effective in extracting complex nonlinear relationships between high-dimensional data, and they are prone to overfitting when data noise exists. Convolutional neural networks (Yann, L., et al., 1998) model the spatial correlations of data through operations such as convolution and pooling, but they are highly dynamic and easily affected by multiple features (Liu, Y., et al., 2016; Shao, Q., et al., 2022; Zhong, J., et al., 2022). Recurrent neural networks (Tomáš, M., et al., 2011; Yang, J., et al., 2022) can be used to extract historical patterns in spatiotemporal data by updating the model parameters by a backpropagation algorithm. Most existing studies use recurrent neural networks to capture a single temporal correlation (Yao, H., et al., 2019; Wang, Y., & Liu, S., 2022; Hou, C., et al., 2021) without considering the compounding effects of periodicity and abrupt variability on the data, making insufficient use of historical data.

Data in real scenarios such as smart cities and smart manufacturing have the following characteristics. In terms of sampling, the data quality is poor and locally sparse owing to the real-world environment and limited cost. For the spatial dimension, there exist correlations between both features and spatiotemporal objects. For the temporal dimension, the spatiotemporal data have both periodicity and mutability, while historical data of different granularities have different effects on the prediction results. These characteristics make it difficult for existing models to accurately model the evolution process of complex spatiotemporal data.

Complete Article List

Search this Journal:
Reset
Volume 20: 1 Issue (2024)
Volume 19: 6 Issues (2023)
Volume 18: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 17: 4 Issues (2021)
Volume 16: 4 Issues (2020)
Volume 15: 4 Issues (2019)
Volume 14: 4 Issues (2018)
Volume 13: 4 Issues (2017)
Volume 12: 4 Issues (2016)
Volume 11: 4 Issues (2015)
Volume 10: 4 Issues (2014)
Volume 9: 4 Issues (2013)
Volume 8: 4 Issues (2012)
Volume 7: 4 Issues (2011)
Volume 6: 4 Issues (2010)
Volume 5: 4 Issues (2009)
Volume 4: 4 Issues (2008)
Volume 3: 4 Issues (2007)
Volume 2: 4 Issues (2006)
Volume 1: 4 Issues (2005)
View Complete Journal Contents Listing