Improvement of multi layer perceptron performance using combination of gradient descent and harmony search for prediction of ground water level

Won Jin Lee; Eui Hoon Lee

doi:10.3741/JKWRA.2022.55.11.903

All Issue

2022 Vol.55, Issue 11 Preview Page Next Page

Research Article

Improvement of multi layer perceptron performance using combination of gradient descent and harmony search for prediction of ground water level 지하수위 예측을 위한 경사하강법과 화음탐색법의 결합을 이용한 다층퍼셉트론 성능향상

30 November 2022. pp. 903-911

PDF XML

Abstract

Groundwater, one of the resources for supplying water, fluctuates in water level due to various natural factors. Recently, research has been conducted to predict fluctuations in groundwater levels using Artificial Neural Network (ANN). Previously, among operators in ANN, Gradient Descent (GD)-based Optimizers were used as Optimizer that affect learning. GD-based Optimizers have disadvantages of initial correlation dependence and absence of solution comparison and storage structure. This study developed Gradient Descent combined with Harmony Search (GDHS), a new Optimizer that combined GD and Harmony Search (HS) to improve the shortcomings of GD-based Optimizers. To evaluate the performance of GDHS, groundwater level at Icheon Yullhyeon observation station were learned and predicted using Multi Layer Perceptron (MLP). Mean Squared Error (MSE) and Mean Absolute Error (MAE) were used to compare the performance of MLP using GD and GDHS. Comparing the learning results, GDHS had lower maximum, minimum, average and Standard Deviation (SD) of MSE than GD. Comparing the prediction results, GDHS was evaluated to have a lower error in all of the evaluation index than GD.

Keywords

Prediction of ground water level

Multi layer perceptron

Optimizer

Gradient descent

Gradient descent combined with harmony search

물을 공급하기 위한 자원 중 하나인 지하수는 다양한 자연적 요인에 의해 수위의 변동이 발생한다. 최근, 인공신경망을 이용하여 지하수위의 변동을 예측하는 연구가 진행되었다. 기존에는 인공신경망 연산자 중 학습에 영향을 미치는 Optimizer로 경사하강법(Gradient Descent, GD) 기반 Optimizer를 사용하였다. GD 기반 Optimizer는 초기 상관관계 의존성과 해의 비교 및 저장 구조 부재의 단점이 존재한다. 본 연구는 GD 기반 Optimizer의 단점을 개선하기 위해 GD와 화음탐색법(Harmony Search, HS)를 결합한 새로운 Optimizer인 Gradient Descent combined with Harmony Search(GDHS)를 개발하였다. GDHS의 성능을 평가하기 위해 다층퍼셉트론(Multi Layer Perceptron, MLP)을 이용하여 이천율현 관측소의 지하수위를 학습 및 예측하였다. GD 및 GDHS를 사용한 MLP의 성능을 비교하기 위해 Mean Squared Error(MSE) 및 Mean Absolute Error(MAE)를 사용하였다. 학습결과를 비교하면, GDHS는 GD보다 MSE의 최대값, 최소값, 평균값 및 표준편차가 작았다. 예측결과를 비교하면, GDHS는 GD보다 모든 평가지표에서 오차가 작은 것으로 평가되었다.

키워드

지하수위 예측

다층퍼셉트론

Optimizer

경사하강법

Gradient Descent combined with Harmony Search

References

Affandia, AK., Watanabe, K., and Tirtomihardjo, H. (2007). "Application of an artificial neural network to estimate groundwater level fluctuation." Journal of Spatial Hydrology, Vol. 7, No. 2, pp. 17-32.

Aqil, M., Kita, I., Yano, A., and Nishiyama, S. (2007). "A comparative study of artificial neural networks and neuro-fuzzy in continuous modeling of the daily and hourly behaviour of runoff." Journal of Hydrology, Vol. 337, No. 1-2, pp. 22-34. 10.1016/j.jhydrol.2007.01.013

Barthel, R., and Banzhaf, S. (2016). "Groundwater and surface water interaction at the regional-scale - a review with focus on regional integrated models." Water Resources Management, Vol. 30, No. 1, pp. 1-32. 10.1007/s11269-015-1163-z

Batelaan, O., De Smedt, F., and Triest, L. (2003). "Regional groundwater discharge: phreatophyte mapping, groundwater modelling and impact analysis of land-use change." Journal of Hydrology, Vol. 275, No. 1-2, pp. 86-108. 10.1016/S0022-1694(03)00018-0

Daliakopoulos, I.N., Coulibaly, P., and Tsanis, I.K. (2005). "Groundwater level forecasting using artificial neural networks." Journal of Hydrology, Vol. 309, No. 1-4, pp. 229-240. 10.1016/j.jhydrol.2004.12.001

Derbela, M., and Nouiri, I. (2020). "Intelligent approach to predict future groundwater level based on artificial neural networks (ANN)." Euro-Mediterranean Journal for Environmental Integration, Vol. 5, No. 3, pp. 1-11. 10.1007/s41207-020-00185-9

Ebrahimi, H., and Rajaee, T. (2017). "Simulation of groundwater level variations using wavelet combined with neural network, linear regression and support vector machine." Global and Planetary Change, Vol. 148, pp. 181-191. 10.1016/j.gloplacha.2016.11.014

Geem, Z.W., Kim, J.H., and Loganathan, G.V. (2001). "A new heuristic optimization algorithm: harmony search." Simulation, Vol. 76, No. 2, pp. 60-68. 10.1177/003754970107600201

Goldberg, D.E., and Holland, J.H. (1988). "Genetic algorithms and machine learning." Machine Learning, Vol. 3, pp. 95-99. 10.1023/A:1022602019183

Joo, G., Park, C., and Im, H. (2020). "Performance evaluation of machine learning optimizers." Journal of Korean Electrical and Electronics Engineers, Vol. 24, No. 3, pp. 766-776.

Kennedy, J., and Eberhart, R. (1995). "Particle swarm optimization." Proceedings of the IEEE International Conference on Neural Networks, Indianapolis, IN, Vol. 4, pp. 1942-1948.

Kim, I., and Lee, J. (2018). "Prediction model for spatial and temporal variation of groundwater level based on river stage." Journal of Hydrologic Engineering, Vol. 23, No. 6, 06018002. 10.1061/(ASCE)HE.1943-5584.0001658

Kim, I., Lee, J., Kim, J., Lee, H., and Lee, J. (2021). "Analysis of groundwater level prediction performance with influencing factors by artificial neural network." Journal of the Korean Geotechnical Society, Vol. 37, No. 5, pp. 19-31.

Kim, Y.N., and Lee, E.H. (2020). "Development of the meta-heuristic optimization algorithm: Exponential bandwidth harmony search with centralized global search." Journal of the Korea Academia-Industrial cooperation Society, Vol. 21, No. 2, pp. 8-18.

Knotters, M., and Bierkens, M.F. (2000). "Physical basis of time series models for water table depths." Water Resources Research, Vol. 36, No. 1, pp. 181-188. 10.1029/1999WR900288

Liu, Q., Jian, W., and Nie, W. (2021). "Rainstorm-induced landslides early warning system in mountainous cities based on groundwater level change fast prediction." Sustainable Cities and Society, Vol. 69, 102817. 10.1016/j.scs.2021.102817

Mahdavi, M., Fesanghary, M., and Damangir, E. (2007). "An improved harmony search algorithm for solving optimization problems." Applied mathematics and computation, Vol. 188, No. 2, pp. 1567-1579. 10.1016/j.amc.2006.11.033

Maxwell, R.M., Condon, L.E., and Kollet, S.J. (2015). "A high-resolution simulation of groundwater and surface water over most of the continental US with the integrated hydrologic model ParFlow v3." Geoscientific Model Development, Vol. 8, No. 3, pp. 923-937. 10.5194/gmd-8-923-2015

McCulloch, W.S., and Pitts, W. (1943). "A logical calculus of the ideas immanent in nervous activity." The Bulletin of Mathematical Biophysics, Vol. 5, No. 4, pp. 115-133. 10.1007/BF02478259

Ministry of Land, Infrastructure and Transport (MOLIT) (2011). Bok-stream basic plan (change) report, p. 34.

Park, C., and Chung, I.M. (2020). "Evaluating the groundwater prediction using LSTM model." Journal of Korea Water Resources Association, Vol. 53, No. 4, pp. 273-283. 10.3741/JKWRA.2020.53.4.273

Rosenblatt, F. (1958). "The perceptron: a probabilistic model for information storage and organization in the brain." Psychological Review, Vol. 65, No. 6, 386.

Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986). "Learning representations by back-propagating errors." Nature, Vol. 323, No. 6088, pp. 533-536. 10.1038/323533a0

Ryu, Y.M., and Lee, E.H. (2022). "Application of neural networks to predict Daecheong Dam water levels." Journal of the Korean Society of Hazard Mitigation, Vol. 22, No. 1, pp. 67-78. 10.9798/KOSHAM.2022.22.1.67

Sahoo, S., and Jha, M.K. (2013). "Groundwater-level prediction using multiple linear regression and artificial neural network techniques: a comparative assessment." Hydrogeology Journal, Vol. 21, No. 8, pp. 1865-1887. 10.1007/s10040-013-1029-5

Sahoo, S., Russo, T.A., Elliott, J., and Foster, I. (2017). "Machine learning algorithms for modeling groundwater level changes in agricultural regions of the US." Water Resources Research, Vol. 53, No. 5, pp. 3878-3895. 10.1002/2016WR019933

Sattari, M.T., Mirabbasi, R., Sushab, R.S., and Abraham, J. (2018). "Prediction of groundwater level in Ardebil plain using support vector regression and M5 tree model." Groundwater, Vol. 56, No. 4, pp. 636-646. 10.1111/gwat.1262029193047

Sedki, A., Ouazar, D., and El Mazoudi, E. (2009). "Evolving neural network using real coded genetic algorithm for daily rainfall - runoff forecasting." Expert Systems with Applications, Vol. 36, No. 3, pp. 4523-4527. 10.1016/j.eswa.2008.05.024

Shin, M.J., Moon, S.H., Kang, K.G., Moon, D.C., and Koh, H.J. (2020). "Analysis of groundwater level variations caused by the changes in groundwater withdrawals using long short-term memory network." Hydrology, Vol. 7, No. 3, 64. 10.3390/hydrology7030064

Suryanarayana, C., Sudheer, C., Mahammood, V., and Panigrahi, B.K. (2014). "An integrated wavelet-support vector machine for groundwater level prediction in Visakhapatnam, India." Neurocomputing, Vol. 145, pp. 324-335. 10.1016/j.neucom.2014.05.026

Trichakis, I.C., Nikolos, I.K., and Karatzas, G.P. (2011). "Artificial neural network (ANN) based modeling for karstic groundwater level simulation." Water Resources Management, Vol. 25, No. 4, pp. 1143-1152. 10.1007/s11269-010-9628-6

White, J.T., Knowling, M.J., and Moore, C.R. (2020). "Consequences of groundwater-model vertical discretization in risk-based decision-Making." Groundwater, Vol. 58, No. 5, pp. 695-709. 10.1111/gwat.1295731667821

Yoo, Y., Kim, D., and Lee, J. (2020). "Performance analysis of various activation functions in super resolution model." Proceedings of the Korea Information Processing Society Conference, KIPS, Vol. 27, No. 1, pp. 504-507.

Yousefi, H., Zahedi, S., Niksokhan, M.H., and Momeni, M. (2019). "Ten-year prediction of groundwater level in Karaj plain (Iran) using MODFLOW2005-NWT in MATLAB." Environmental Earth Sciences, Vol. 78, No. 12, pp. 1-14. 10.1007/s12665-019-8340-y

Information

Publisher :KOREA WATER RESOURECES ASSOCIATION
Publisher(Ko) :한국수자원학회
Journal Title :Journal of Korea Water Resources Association
Journal Title(Ko) :한국수자원학회 논문집
Volume : 55
No :11
Pages :903-911
Received Date : 2022-08-26
Revised Date : 2022-10-07
Accepted Date : 2022-10-13
DOI :https://doi.org/10.3741/JKWRA.2022.55.11.903

Journal of Korea Water Resources Association ISSN:2799-8746(Print) 2799-8754(Online) 한국수자원학회 논문집

All Issue