Abstract
Wind dynamics are extremely complex and have critical impacts on the level of damage from natural hazards, such as storms and wildfires. In the wake of climate change, wind dynamics are becoming more complex, making the prediction of future wind characteristics a more challenging task. Nevertheless, having long-term projections of some wind characteristics, such as daily wind speed, is crucial for effective monitoring of climate change, and for efficient disaster risk management. Furthermore, accurate projections of wind speed result in optimized generation of wind-based electric power. General circulation models (GCMs) provide long-term simulations (often till year 2100 or more) of multiple climate variables. However, simulations from a GCM are at a grid with coarse spatial resolution, rendering them ineffective to resolve and analyze climate change at the local regional level. Spatial downscaling techniques are often used to map such global large-scale simulations to a local small-scale region. In this paper, we present a novel deep learning framework for spatial downscaling, specifically for forecasting the daily average wind speed at a local station level using GCM simulations. Our framework, named wind convolutional neural networks with transformers, or WCT for short, consists of multi-head convolutional neural networks, followed by stacked transformers, and an uncertainty quantification component based on Bayesian inference. Experimental results show the suitability of WCT when applied on four wind stations in New Jersey and Pennsylvania, USA. Moreover, we use the trained WCT on future GCM simulations to produce local-scale daily wind speed projections up to the year 2100.
Similar content being viewed by others
References
Parry, M., Parry, M.L., Canziani, O., Palutikof, J., Van der Linden, P., Hanson, C.: Climate Change 2007-Impacts, Adaptation and Vulnerability: Working Group II Contribution to the Fourth Assessment Report of the the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge (2007)
Stocker, T., Qin, D., Plattner, G., Tignor, M., Allen, S., Boschung, J., Nauels, A., Xia, Y., Bex, V., Midgley, P.: IPCC, 2013: Summary for Policymakers in Climate Change 2013: the Physical Science Basis, Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge (2013)
USGCRP.: Climate Science Special Report: Fourth National Climate Assessment, Volume I. U.S. Global Change Research Program (2017). https://doi.org/10.7930/J0J964J6
Murawski, A., Bürger, G., Vorogushyn, S., Merz, B.: Can local climate variability be explained by weather patterns? A multi-station evaluation for the Rhine basin. Hydrol. Earth Syst. Sci. 20(10), 4283–4306 (2016)
El-Samra, R., Bou-Zeid, E., Bangalath, H.K., Stenchikov, G., El-Fadel, M.: Seasonal and regional patterns of future temperature extremes: high-resolution dynamic downscaling over a complex terrain. J. Geophys. Res. Atmos. 123(13), 6669–6689 (2018)
Druyan, L.M., Fulakeza, M., Lonergan, P.: Dynamic downscaling of seasonal climate predictions over Brazil. J. Clim. 15(23), 3411–3426 (2002)
Xu, Z., Yang, Z.-L.: An improved dynamical downscaling method with GCM bias corrections and its validation with 30 years of climate simulations. J. Clim. 25(18), 6271–6286 (2012)
Stoner, A.M., Hayhoe, K., Yang, X., Wuebbles, D.J.: An asynchronous regional regression model for statistical downscaling of daily climate variables. Int. J. Climatol. 33(11), 2473–2494 (2013)
Li, M., Zhang, F., Barnes, S., Wang, X.: Assessing storm surge impacts on coastal inundation due to climate change: case studies of Baltimore and Dorchester County in Maryland. Nat. Hazards 103, 2561–2588 (2020)
Ghosh, S., Mujumdar, P.P.: Statistical downscaling of GCM simulations to streamflow using relevance vector machine. Adv. Water Resour. 31(1), 132–146 (2008)
Wilby, R.L., Wigley, T., Conway, D., Jones, P., Hewitson, B., Main, J., Wilks, D.: Statistical downscaling of general circulation model output: a comparison of methods. Water Resour. Res. 34(11), 2995–3008 (1998)
von Storch, H.: Inconsistencies at the interface of climate impact studies and global climate research. Meteorol. Z. 4, 72–80 (1995)
von Storch, H.: The global and regional climate system. In: von Storch, H., Flosser, G. (eds.) Anthropogenic Climate Change. Springer, pp. 3–36 (1999)
He, X., Chaney, N.W., Schleiss, M., Sheffield, J.: Spatial downscaling of precipitation using adaptable random forests. Water Resour. Res. 52(10), 8217–8237 (2016)
Sachindra, D., Ahmed, K., Rashid, M.M., Shahid, S., Perera, B.: Statistical downscaling of precipitation using machine learning techniques. Atmos. Res. 212, 240–258 (2018)
Li, X., Li, Z., Huang, W., Zhou, P.: Performance of statistical and machine learning ensembles for daily temperature downscaling. Theor. Appl. Climatol. 140(1), 571–588 (2020)
Hüllermeier, E., Waegeman, W.: Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach. Learn. 110(3), 457–506 (2021)
Nguyen, V.-L., Destercke, S., Masson, M.-H., Hüllermeier, E.: Reliable multi-class classification based on pairwise epistemic and aleatoric uncertainty. In: 27th International Joint Conference on Artificial Intelligence (IJCAI 2018), pp. 5089–5095 (2018)
Jiang, H., Jing, J., Wang, J., Liu, C., Li, Q., Xu, Y., Wang, J.T.L., Wang, H.: Tracing Hα fibrils through Bayesian deep learning. Astrophys. J. Suppl. Ser. 256(1), 20 (2021)
Myojin, T., Hashimoto, S., Ishihama, N.: Detecting uncertain BNN outputs on FPGA using Monte Carlo dropout sampling. In: International Conference on Artificial Neural Networks. Springer, pp. 27-38 (2020)
Myojin, T., Hashimoto, S., Mori, K., Sugawara, K., Ishihama, N.: Improving reliability of object detection for lunar craters using Monte Carlo dropout. In: International Conference on Artificial Neural Networks. Springer, pp. 68-80 (2019)
Gerges, F., Boufadel, M.C., Bou-Zeid, E., Nassif, H., Wang, J.T.L.: A novel Bayesian deep learning approach to the downscaling of wind speed with uncertainty quantification. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 55–66 (2022)
IPCC: Climate Change 2013: the Physical Science Basis (Fifth Assessment Report). United Nations, New York (2013)
Chen, S.-T., Yu, P.-S., Tang, Y.-H.: Statistical downscaling of daily precipitation using support vector machines and multivariate analysis. J. Hydrol. 385(1–4), 13–22 (2010)
Pang, B., Yue, J., Zhao, G., Xu, Z.: Statistical downscaling of temperature with the random forest model. Adv. Meteorol. 2017 (2017). https://doi.org/10.1155/2017/7265178
Hutengs, C., Vohland, M.: Downscaling land surface temperatures at regional scales with random forest regression. Remote Sens. Environ. 178, 127–141 (2016)
Wu, H., Li, W.: Downscaling land surface temperatures using a random forest regression model with multitype predictor variables. IEEE Access 7, 21904–21916 (2019)
Pour, S.H., Shahid, S., Chung, E.-S., Wang, X.-J.: Model output statistics downscaling using support vector machine for the projection of spatial and temporal changes in rainfall of Bangladesh. Atmos. Res. 213, 149–162 (2018)
Tripathi, S., Srinivas, V., Nanjundiah, R.S.: Downscaling of precipitation for climate change scenarios: a support vector machine approach. J. Hydrol. 330(3–4), 621–640 (2006)
Ghosh, S., Mujumdar, P.: Future rainfall scenario over Orissa with GCM projections by statistical downscaling. Curr. Sci. 90(3), 396–404 (2006)
Sachindra, D., Huang, F., Barton, A., Perera, B.: Least square support vector and multi-linear regression for statistically downscaling general circulation model outputs to catchment streamflows. Int. J. Climatol. 33(5), 1087–1106 (2013)
Xu, R., Chen, N., Chen, Y., Chen, Z.: Downscaling and projection of multi-CMIP5 precipitation using machine learning methods in the upper Han River Basin. Adv. Meteorol. 2020 (2020). https://doi.org/10.1155/2020/8680436
Kannan, S., Ghosh, S.: Prediction of daily rainfall state in a river basin using statistical downscaling from GCM output. Stoch. Env. Res. Risk Assess. 25(4), 457–474 (2011)
Sun, L., Lan, Y.: Statistical downscaling of daily temperature and precipitation over China using deep learning neural models: localization and comparison with other methods. Int. J. Climatol. 41(2), 1128–1147 (2021)
Yang, Z., Wang, Y., Liu, C., Chen, H., Xu, C., Shi, B., Xu, C., Xu, C.: Legonet: efficient convolutional neural networks with lego filters. In: International Conference on Machine Learning. PMLR, pp. 7005–7014 (2019)
Pan, X., Shi, J., Luo, P., Wang, X., Tang, X.: Spatial as deep: spatial CNN for traffic scene understanding. In: Proceedings of the 32nd Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (2018)
Jin, C., Liang, H., Chen, D., Lin, Z., Wu, M.: Identifying mobility of drug addicts with multilevel spatial-temporal convolutional neural network. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, pp. 477-488 (2019)
Liu, Z., Wan, M., Guo, S., Achan, K., Yu, P.S.: Basconv: aggregating heterogeneous interactions for basket recommendation with graph convolutional neural network. In: Proceedings of the 2020 Society for Industrial and Applied Mathematics (SIAM) International Conference on Data Mining, pp. 64–72 (2020)
Baño-Medina, J., Manzanas, R., Gutiérrez, J.M.: Configuration and intercomparison of deep learning neural models for statistical downscaling. Geosci. Model Dev. 13(4), 2109–2124 (2020)
Maraun, D., Widmann, M., Gutiérrez, J.M., Kotlarski, S., Chandler, R.E., Hertig, E., Wibig, J., Huth, R., Wilcke, R.A.: VALUE: a framework to validate downscaling approaches for climate change studies. Earth’s Future 3(1), 1–14 (2015)
Misra, S., Sarkar, S., Mitra, P.: Statistical downscaling of precipitation using long short-term memory recurrent neural networks. Theor. Appl. Climatol. 134(3), 1179–1196 (2018)
Guo, T., Lin, T., Antulov-Fantulin, N.: Exploring interpretable LSTM neural networks over multi-variable data. In: International Conference on Machine Learning. PMLR, pp. 2494–2504 (2019)
Hu, Z., Turki, T., Phan, N., Wang, J.T.L.: A 3D atrous convolutional long short-term memory network for background subtraction. IEEE Access 6, 43450–43459 (2018)
Liu, H., Liu, C., Wang, J.T.L., Wang, H.: Predicting solar flares using a long short-term memory network. Astrophys. J. 877(2), 121 (2019)
Segovia-Dominguez, I., Zhen, Z., Wagh, R., Lee, H., Gel, Y.R.: TLife-LSTM: forecasting future COVID-19 progression with topological signatures of atmospheric conditions. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 201–212. Springer (2021)
Shalaby, M., Stutzki, J., Schubert, M., Günnemann, S.: An LSTM approach to patent classification based on fixed hierarchy vectors. In: Proceedings of the 2018 Society for Industrial and Applied Mathematics (SIAM) International Conference on Data Mining, pp. 495–503 (2018)
Gerges, F., Boufadel, M.C., Bou-Zeid, E., Nassif, H., Wang, J.T.L.: A novel deep learning approach to the statistical downscaling of temperatures for monitoring climate change. In: The 6th International Conference on Machine Learning and Soft Computing, Haikou, China, pp. 1–7 (2022)
Liu, G., Zhang, R., Hang, R., Ge, L., Shi, C., Liu, Q.: Statistical downscaling of temperature distributions in southwest China by using terrain-guided attention network. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 16, 1678–1690 (2023)
Yang, D., Liu, S., Hu, Y., Liu, X., Xie, J., Zhao, L.: Predictor selection for CNN-based statistical downscaling of monthly precipitation. Adv. Atmos. Sci. 40, 1117–1131 (2023)
Nourani, V., Sadikoglu, F., Khodkar, K., Shahli, A.: Downscaling precipitation from GCM parameters using recurrent neural networks. In: 11th International Conference on Theory and Application of Soft Computing, Computing with Words and Perceptions and Artificial Intelligence-ICSCCW-2021 11 , pp. 294–301. Springer (2022)
Gerges, F., Boufadel, M.C., Bou-Zeid, E., Nassif, H., Wang, J.T.L.: Deep learning-based downscaling of temperatures for monitoring local climate change using global climate simulation data. World Sci. Annu. Rev. Artif. Intell. (2023). https://doi.org/10.1142/s2811032322500011
Gerges, F., Boufadel, M.C., Bou-Zeid, E., Darekar, A., Nassif, H., Wang, J.T.L.: Bayesian multi-head convolutional neural networks with Bahdanau attention for forecasting daily precipitation in climate change monitoring. In:European Confrence of Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2022, Part V, pp. 565–580. Springer, (2023)
Quistgaard, T., Langen, P.L., Denager, T., Schneider, R., Stisen, S.: Deep Learning guided statistical downscaling of climate projections for use in hydrological impact modeling in Danish peatlands. Copernicus Meetings (2023)
Nguyen, D.T., Ashraf, S., Le, M., Ali, M.: Projection of climate variables by general circulation and deep learning model for Lahore, Pakistan. Ecol. Inform. 75, 102077 (2023)
Menne, M.J., Durre, I., Vose, R.S., Gleason, B.E., Houston, T.G.: An overview of the global historical climatology network-daily database. J. Atmos. Ocean. Tech. 29(7), 897–910 (2012)
Griffies, S.M., Winton, M., Donner, L.J., Horowitz, L.W., Downes, S.M., Farneti, R., Gnanadesikan, A., Hurlin, W.J., Lee, H.-C., Liang, Z.: The GFDL CM3 coupled climate model: characteristics of the ocean and sea ice simulations. J. Clim. 24(13), 3520–3544 (2011)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Arık, S.Ö., Jun, H., Diamos, G.: Fast spectrogram inversion using multi-head convolutional neural networks. IEEE Signal Process. Lett. 26(1), 94–98 (2018)
Khan, Z.N., Ahmad, J.: Attention induced multi-head convolutional neural network for human activity recognition. Appl. Soft Comput. 110, 107671 (2021)
Linmans, J., Laak, J.V.D., Litjens, G.: Efficient out-of-distribution detection in digital pathology using multi-head convolutional neural networks. In: Proceedings of the Third Conference on Medical Imaging with Deep Learning, vol. 121, pp. 465–478. PMLR (2020)
Devika, R., Vairavasundaram, S., Mahenthar, C.S.J., Varadarajan, V., Kotecha, K.: A deep learning model based on BERT and sentence transformer for semantic keyphrase extraction on big social data. IEEE Access 9, 165252–165261 (2021)
Martín-Gutiérrez, D., Hernández-Peñaloza, G., Hernández, A.B., Lozano-Diez, A., Álvarez, F.: A deep learning approach for robust detection of bots in twitter using transformers. IEEE Access 9, 54591–54601 (2021)
Aurpa, T.T., Sadik, R., Ahmed, M.S.: Abusive Bangla comments detection on Facebook using transformer-based deep learning models. Soc. Netw. Anal. Min. 12(1), 1–14 (2022)
Saltz, P., Lin, S.Y., Cheng, S.C., Si, D.: Dementia detection using transformer-based deep learning and natural language processing models. In: Proceedings of the 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI), pp. 509–510 (2021)
Parmar, N., Vaswani, A., Uszkoreit, J., Kaiser, L., Shazeer, N., Ku, A., Tran, D.: Image transformer. In: International Conference on Machine Learning, pp. 4055–4064. PMLR (2018)
Cai, T., Shen, M., Peng, H., Jiang, L., Dai, Q.: Improving transformer with sequential context representations for abstractive text summarization. In: International Conference on Natural Language Processing and Chinese Computing, pp. 512–524 (2019)
Guo, D., Terzopoulos, D.: A transformer-based network for anisotropic 3D medical image segmentation. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 8857–8861 (2021)
Liu, W., Li, C., Rahaman, M.M., Jiang, T., Sun, H., Wu, X., Hu, W., Chen, H., Sun, C., Yao, Y.: Is the aspect ratio of cells important in deep learning? A robust comparison of deep learning methods for multi-scale cytopathology cell image classification: from convolutional neural networks to visual transformers. Comput. Biol. Med. 141, 105026 (2022)
Wang, H., Ji, Y., Song, K., Sun, M., Lv, P., Zhang, T.: ViT-P: classification of genitourinary syndrome of menopause from OCT images based on vision transformer models. IEEE Trans. Instrum. Meas. 70, 1–14 (2021)
Park, S., Kim, G., Oh, Y., Seo, J.B., Lee, S.M., Kim, J.H., Moon, S., Lim, J.-K., Ye, J.C.: Multi-task vision transformer using low-level chest X-ray feature corpus for COVID-19 diagnosis and severity quantification. Med. Image Anal. 75, 102299 (2022)
Ikromjanov, K., Bhattacharjee, S., Hwang, Y.-B., Sumon, R. I., Kim, H.-C., Choi, H.-K.: Whole slide image analysis and detection of prostate cancer using vision transformers. In: Proceedings of the 2022 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pp. 399–402 (2022)
Shen, L., Wang, Y.: TCCT: tightly-coupled convolutional transformer on time series forecasting. Neurocomputing 480, 131–145 (2022)
Abbasi, A.R., Mahmoudi, M.R., Arefi, M.M.: Transformer winding faults detection based on time series analysis. IEEE Trans. Instrum. Meas. 70, 1–10 (2021)
Narayan, A., Mishra, B.S., Hiremath, P.S., Pendari, N.T., Gangisetty, S.: An ensemble of transformer and LSTM approach for multivariate time series data classification. In: Proceedings of the 2021 IEEE International Conference on Big Data (Big Data), pp. 5774–5779 (2021)
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., Eickhoff, C.: A transformer-based framework for multivariate time series representation learning. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2114–2124 (2021)
Mohammdi Farsani, R., Pazouki, E.: A transformer self-attention model for time series forecasting. J. Electr. Comput. Eng. Innov. (JECEI) 9(1), 1–10 (2021)
Wu, N., Green, B., Ben, X., O'Banion, S.: Deep transformer models for time series forecasting: the influenza prevalence case.arXiv:2001.08317 (2020)
Cai, L., Janowicz, K., Mai, G., Yan, B., Zhu, R.: Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting. Trans. GIS 24(3), 736–755 (2020)
Zhang, K., Hawkins, C., Zhang, Z.: General-purpose Bayesian tensor learning with automatic rank determination and uncertainty quantification. Front. Artif. Intell. 4, 668353 (2021)
Liu, J.: Variable selection with rigorous uncertainty quantification using deep Bayesian neural networks: posterior concentration and Bernstein-von Mises phenomenon. In: International Conference on Artificial Intelligence and Statistics, 2021: Proceedings of Machine Learning Research (PMLR), pp. 3124–3132
Wang, Y., Rocková, V.: Uncertainty quantification for sparse deep learning. In: International Conference on Artificial Intelligence and Statistics, 2020: Proceedings of Machine Learning Research (PMLR), pp. 298–308
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In: International Conference on Machine Learning, 2016: Proceedings of Machine Learning Research (PMLR), pp. 1050–1059
Roy, S., Menapace, W., Oei, S., Luijten, B., Fini, E., Saltori, C., Huijben, I., Chennakeshava, N., Mento, F., Sentelli, A.: Deep learning for classification and localization of COVID-19 markers in point-of-care lung ultrasound. IEEE Trans. Med. Imaging 39(8), 2676–2687 (2020)
Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. J. Am. Stat. Assoc. 112(518), 859–877 (2017)
Kwon, Y., Won, J.-H., Kim, B.J., Paik, M.C.: Uncertainty quantification using Bayesian neural networks in classification: application to biomedical image segmentation. Comput. Stat. Data Anal. 142, 106816 (2020)
Nash, J.E., Sutcliffe, J.V.: River flow forecasting through conceptual models part I—a discussion of principles. J. Hydrol. 10(3), 282–290 (1970)
Ahmed, K., Shahid, S., Haroon, S.B., Xiao-Jun, W.: Multilayer perceptron neural network for downscaling rainfall in arid region: a case study of Baluchistan, Pakistan. J. Earth Syst. Sci. 124(6), 1325–1341 (2015)
Ghosh, S.: SVM‐PGSL coupled approach for statistical downscaling to predict rainfall from GCM output. J. Geophys. Res. Atmos. 115 (2010). https://doi.org/10.1029/2009JD013548
Solé, X., Ramisa, A., Torras, C.: Evaluation of random forests on large-scale classification problems using a bag-of-visual-words representation. In: Artificial Intelligence Research and Development. Amsterdam, the Netherlands: IOS Press, pp. 273–276 (2014)
Tsironi, E., Barros, P., Weber, C., Wermter, S.: An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition. Neurocomputing 268, 76–86 (2017)
Funding
Funding for this study was provided by the Bridge Resource Program (BRP) from the New Jersey Department of Transportation.
Author information
Authors and Affiliations
Contributions
FG wrote the paper, gathered the data, wrote the machine learning scripts and conducted the experimentations. BF, HN and EBZ provided GCM and results interpretation and contributed to the development of the predictive model through environmental domain expertise. JW supervised the research, interpreted the data, contributed to the writing and to the development of predictive models.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or nonfinancial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gerges, F., Boufadel, M.C., Bou-Zeid, E. et al. Downscaling daily wind speed with Bayesian deep learning for climate monitoring. Int J Data Sci Anal (2023). https://doi.org/10.1007/s41060-023-00397-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s41060-023-00397-6