Downloads: 3 | Views: 63 | Weekly Hits: ⮙2 | Monthly Hits: ⮙2

Doctoral Thesis | Earth and Planetary Science | India | Volume 12 Issue 11, November 2023 | Rating: 5.5 / 10

Time Series Forecasting of Air Pollutant PM2.5 Using Transformer Architecture

K. Azhahudurai | Dr. V. Veeramanikandan

Abstract: Transformer architectures are widely used, especially in computer vision and natural language processing. Transformers have been used recently in a number of time-series analysis applications. An overview of the Transformer architecture and its uses in time-series analysis is given in the literature review. To improve performance, the Transformer's primary parts?the encoder/decoder, multi-head, positional encoding, and self-attention mechanism?have been updated. To implement time-series analysis, a few improvements to the original transformer architecture were adopted. Additionally, the optimal hyperparameters values for overcoming the difficulty of successfully training Transformers for time-series analysis are provided in this work. The effectiveness of the Transformer model in forecasting PM2.5 concentrations is examined in this paper. The dataset is pre-processed as a first step. In order to minimize the input parameters while taking into account their statistical significance, multi-collinearity among the independent variables is found using a Variance Inflation Factor (VIF). The proposed model have been trained to forecast PM2.5 concentrations up to one day ahead of time.

Keywords: transformer architecture, time series analysis, self attention, hyperparameters, forecasting PM 2.5, multi-collinearity, Variance Inflation Factor

Edition: Volume 12 Issue 11, November 2023,

Pages: 2075 - 2082