Journal of Guangdong University of Technology ›› 2021, Vol. 38 ›› Issue (06): 1-8.doi: 10.12052/gdutxb.210109

    Next Articles

An Evolutionary Optimization of LSTM for Model Recovery of Geophysical Fluid Dynamics

Gary Yen1, Li Bo2, Xie Sheng-li2   

  1. 1. School of Electrical and Computer Engineering, Oklahoma State University, Stillwater 74078, USA;
    2. School of Automation, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2021-07-22 Online:2021-11-10 Published:2021-11-09

HTML

PDF (PC)

2599

Abstract: The computational models for geophysical fluid dynamics are computationally enormously expensive to employ in tasks such as data assimilation and uncertainty quantification. Naturally, surrogate models seeking to alleviate the computational burden has been proposed. Researchers have started applying artificial intelligence and machine learning algorithms, particularly artificial neural networks, to build data-driven surrogate models for geophysical flows. The performance of the neural network highly relies upon their architecture design and selection of hyper-parameters. These neural networks are usually manually crafted through trial and error to maximize their performance. This often demands specialized knowledge of the underlying neural network as well as the domain problems of interest. This limitation can be addressed by using an evolutionary algorithm to automatically design and select optimal hyper-parameters of the neural network. In this study, the genetic algorithm is applied to effectively design the long short-term memory (LSTM) neural network to build the forecasting model of the temperature in the NOAA sea-surface temperature data set.

Key words: long short-term memory neural networks, genetic algorithms, neural network architecture optimization, deep learning

CLC Number: 

  • TP391.4
[1] LEWIS J M, LAKSHMIVARAHAN S, DHALL S. Dynamic data assimilation: a least squares approach[M]. Cambridge: Cambridge University Press, 2006.
[2] MOIN P, MAHESH K. Direct numerical simulation: a tool in turbulence research [J]. Annual Review of Fluid Mechanics, 1998, 30(1): 539-578.
[3] POLAT O, TUNCER I H. Aerodynamic shape optimization of wind turbine blades using a parallel genetic algorithm [J]. Procedia Engineering, 2013, 61: 28-31.
[4] LUCIA D J, BERAN P S, SILVA W A. Reduced-order modeling: new approaches for computational physics [J]. Progress in Aerospace Sciences, 2004, 40(1-2): 51-117.
[5] TAIRA K, BRUNTON S L, DAWSON S T M, et al. Modal analysis of fluid flows: an overview [J]. Aiaa Journal, 2017, 55(12): 4013-4041.
[6] BENNER P, GUGERCIN S, WILLCOX K. A survey of projection-based model reduction methods for parametric dynamical systems [J]. SIAM Review, 2015, 57(4): 483-531.
[7] XIAO D, FANG F, PAIN C, et al. Non-intrusive reduced-order modelling of the Navier-Stokes equations based on RBF interpolation [J]. International Journal for Numerical Methods in Fluids, 2015, 79(11): 580-595.
[8] HESTHAVEN J S, UBBIALI S. Non-intrusive reduced order modeling of nonlinear problems using neural networks [J]. Journal of Computational Physics, 2018, 363: 55-78.
[9] PAWAR S, RAHMAN S M, VADDIREDDY H, et al. A deep learning enabler for nonintrusive reduced order modeling of fluid flows [J]. Physics of Fluids, 2019, 31(8): 085101.
[10] SCHNEIDER T, LAN S, STUART A, et al. Earth system modeling 2.0: a blueprint for models that learn from observations and targeted high-resolution simulations [J]. Geophysical Research Letters, 2017, 44(24): 12396-12417.
[11] O'GORMAN P A, DWYER J G. Using machine learning to parameterize moist convection: potential for modeling of climate, climate change, and extreme events [J]. Journal of Advances in Modeling Earth Systems, 2018, 10(10): 2548-2563.
[12] REICHSTEIN M, CAMPS-VALLS G, STEVENS B, et al. Deep learning and process understanding for data-driven earth system science [J]. Nature, 2019, 566(7743): 195-204.
[13] MOHAN A T, GAITONDE D V. A deep learning based approach to reduced order modeling for turbulent flow control using LSTM neural networks[EB/OL]. arXiv preprint arXiv: 1804.09269, 2018. [2021-07-02]. http://export.arxiv.org/abs/1804.09269.
[14] RAHMAN S M, PAWAR S, SAN O, et al. Nonintrusive reduced order modeling framework for quasigeostrophic turbulence [J]. Physical Review E, 2019, 100(5): 053306.
[15] TAN C, SUN F, KONG T, et al. A survey on deep transfer learning[C]//International Conference on Artificial Neural Networks. Switzerland: Springer, Cham, 2018: 270-279.
[16] MJOLSNESS E, DECOSTE D. Machine learning for science: state of the art and future prospects [J]. Science, 2001, 293(5537): 2051-2055.
[17] BRUNTON S L, NOACK B R, KOUMOUTSAKOS P. Machine learning for fluid mechanics [J]. Annual Review of Fluid Mechanics, 2020, 52: 477-508.
[18] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks [J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
[19] ELSKEN T, METZEN J H, HUTTER F. Neural architecture search: a survey [J]. The Journal of Machine Learning Research, 2019, 20(1): 1997-2017.
[20] HOLMES P, LUMLEY J L, BERKOOZ G, et al. Turbulence, coherent structures, dynamical systems and symmetry[M]. Cambridge: Cambridge University Press, 2012.
[21] ROWLEY C W, DAWSON S T M. Model reduction for flow analysis and control [J]. Annual Review of Fluid Mechanics, 2017, 49: 387-417.
[22] PATHAK J, HUNT B, GIRVAN M, et al. Model-free prediction of large spatiotemporally chaotic systems from data: a reservoir computing approach [J]. Physical Review Letters, 2018, 120(2): 024102.
[23] VLACHAS P R, BYEON W, WAN Z Y, et al. Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks [J]. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2018, 474(2213): 20170844.
[24] MAULIK R, MOHAN A, LUSCH B, et al. Time-series learning of latent-space dynamics for reduced-order model closure [J]. Physica D: Nonlinear Phenomena, 2020, 405: 132368.
[25] AHMED S E, RAHMAN S M, SAN O, et al. Memory embedded non-intrusive reduced order modeling of non-ergodic flows [J]. Physics of Fluids, 2019, 31(12): 126602.
[26] WAN Z Y, VLACHAS P, KOUMOUTSAKOS P, et al. Data-assisted reduced-order modeling of extreme events in complex dynamical systems [J]. PloS One, 2018, 13(5): e0197704.
[27] HOCHREITER S, SCHMIDHUBER J. Long short-term memory [J]. Neural Computation, 1997, 9(8): 1735-1780.
[28] XIE L, YUILLE A. Genetic CNN[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice: IEEE, 2017: 1379-1388.
[29] SUN Y, XUE B, ZHANG M, et al. Automatically designing CNN architectures using the genetic algorithm for image classification [J]. IEEE Transactions on Cybernetics, 2020, 50(9): 3840-3854.
[30] SUN Y, YEN G G, YI Z. Evolving unsupervised deep neural networks for learning meaningful representations [J]. IEEE Transactions on Evolutionary Computation, 2018, 23(1): 89-103.
[31] SUN Y, WANG H, XUE B, et al. Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor [J]. IEEE Transactions on Evolutionary Computation, 2019, 24(2): 350-364.
[32] SUN Y, XUE B, ZHANG M, et al. Evolving deep convolutional neural networks for image classification [J]. IEEE Transactions on Evolutionary Computation, 2019, 24(2): 394-407.
[33] SUN Y, XUE B, ZHANG M, et al. Completely automated CNN architecture design based on blocks [J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(4): 1242-1254.
[34] ZHOU Y, YEN G G, YI Z. Evolutionary compression of deep neural networks for biomedical image segmentation [J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(8): 2916-2929.
[35] DROZDZAL M, VORONTSOV E, CHARTRAND G, et al. The importance of skip connections in biomedical image segmentation[C]//Deep learning and data labeling for medical applications. Switzerland: Springer, 2016: 179-187.
[36] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 770-778.
[37] MAULIK R, LUSCH B, BALAPRAKASH P. Non-autoregressive time-series methods for stable parametric reduced-order models [J]. Physics of Fluids, 2020, 32(8): 087115.
[1] Liu Dong-ning, Wang Zi-qi, Zeng Yan-jiao, Wen Fu-yan, Wang Yang. Prediction Method of Gene Methylation Sites Based on LSTM with Compound Coding Characteristics [J]. Journal of Guangdong University of Technology, 2023, 40(01): 1-9.
[2] Xu Wei-feng, Cai Shu-ting, Xiong Xiao-ming. Visual Inertial Odometry Based on Deep Features [J]. Journal of Guangdong University of Technology, 2023, 40(01): 56-60,76.
[3] Liu Hong-wei, Lin Wei-zhen, Wen Zhan-ming, Chen Yan-jun, Yi Min-qi. A MABM-based Model for Identifying Consumers' Sentiment Polarity―Taking Movie Reviews as an Example [J]. Journal of Guangdong University of Technology, 2022, 39(06): 1-9.
[4] Zhang Yun, Wang Xiao-dong. A Review and Thinking of Deep Learning with a Restricted Number of Samples [J]. Journal of Guangdong University of Technology, 2022, 39(05): 1-8.
[5] Zheng Jia-bi, Yang Zhen-guo, Liu Wen-yin. Marketing-Effect Estimation Based on Fine-grained Confounder Balancing [J]. Journal of Guangdong University of Technology, 2022, 39(02): 55-61.
[6] Lai Jun, Liu Zhen-yu, Liu Sheng-hai. A Small Sample Data Prediction Method Based on Global Data Shuffling [J]. Journal of Guangdong University of Technology, 2021, 38(03): 17-21.
[7] Cen Shi-jie, He Yuan-lie, Chen Xiao-cong. A Monocular Depth Estimation Combined with Attention and Unsupervised Deep Learning [J]. Journal of Guangdong University of Technology, 2020, 37(04): 35-41.
[8] Zeng Bi, Ren Wan-ling, Chen Yun-hua. An Unpaired Face Illumination Normalization Method Based on CycleGAN [J]. Journal of Guangdong University of Technology, 2018, 35(05): 11-19.
[9] Yang Meng-jun, Su Cheng-yue, Chen Jing, Zhang Jie-xin. Loop Closure Detection for Visual SLAM Using Convolutional Neural Networks [J]. Journal of Guangdong University of Technology, 2018, 35(05): 31-37.
[10] Chen Xu, Zhang Jun, Chen Wen-wei, Li Shuo-hao. Convolutional Neural Network Algorithm and Case [J]. Journal of Guangdong University of Technology, 2017, 34(06): 20-26.
[11] Liu Zhen-yu, Li Jia-jun, Wang Kun. A Fingerprint Matching Localization Method Based on Deep Auto Encoder [J]. Journal of Guangdong University of Technology, 2017, 34(05): 15-21.
[12] HE Yong, WEN Jie-Chang, HUANG Mei-Hua. The Three Layers of Large-scale Emergency Material Allocation Strategy Based on the GA [J]. Journal of Guangdong University of Technology, 2016, 33(02): 37-41.
[13] LI Ling-Yu, GUO Gui-Fa, XU Jin-Biao. Optimizing and Adjusting the Parameter of PID Based on Simulated Annealing Genetic Algorithms [J]. Journal of Guangdong University of Technology, 2010, 27(2): 80-83.
[14] Song Hua-bing1,2,Zhang Xin-zhen2. A Study of Time-delay Uncertain Water Quality Model BOD-DO [J]. Journal of Guangdong University of Technology, 2008, 25(3): 66-69.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © 2017 Journal of Guangdong University of Technology
Add：729 East Dongfeng RD., Guangzhou PRC. Postcode: 510090
Tel：020-37626653,020-37626139,020-37626396
Email：xbzrb@gdut.edu.cn
Supported:Beijing Magtech