Development of new artificial neural network optimizer to improve water quality index prediction performance

Yong Min Ryu; Young Nam Kim; Dae Won Lee; Eui Hoon Lee

doi:10.3741/JKWRA.2024.57.2.73

All Issue

2024 Vol.57, Issue 2 Next Page

Research Article

Development of new artificial neural network optimizer to improve water quality index prediction performance 수질 지수 예측성능 향상을 위한 새로운 인공신경망 옵티마이저의 개발

29 February 2024. pp. 73-85

PDF XML

Abstract

Predicting water quality of rivers and reservoirs is necessary for the management of water resources. Artificial Neural Networks (ANNs) have been used in many studies to predict water quality with high accuracy. Previous studies have used Gradient Descent (GD)-based optimizers as an optimizer, an operator of ANN that searches parameters. However, GD-based optimizers have the disadvantages of the possibility of local optimal convergence and absence of a solution storage and comparison structure. This study developed improved optimizers to overcome the disadvantages of GD-based optimizers. Proposed optimizers are optimizers that combine adaptive moments (Adam) and Nesterov-accelerated adaptive moments (Nadam), which have low learning errors among GD-based optimizers, with Harmony Search (HS) or Novel Self-adaptive Harmony Search (NSHS). To evaluate the performance of Long Short-Term Memory (LSTM) using improved optimizers, the water quality data from the Dasan water quality monitoring station were used for training and prediction. Comparing the learning results, Mean Squared Error (MSE) of LSTM using Nadam combined with NSHS (NadamNSHS) was the lowest at 0.002921. In addition, the prediction rankings according to MSE and R² for the four water quality indices for each optimizer were compared. Comparing the average of ranking for each optimizer, it was confirmed that LSTM using NadamNSHS was the highest at 2.25.

Keywords

Water quality index

Long Short-Term Memory

Novel Self-adaptive Harmony Search

Improved optimizer

하천과 저수지의 수질을 예측하는 것은 수자원관리를 위해 필요하다. 높은 정확도의 수질 예측을 위해 많은 연구들에서 인공신경망이 활용되었다. 기존 연구들은 매개변수를 탐색하는 인공신경망의 연산자인 옵티마이저로 경사하강법 기반 옵티마이저를 사용하였다. 그러나 경사하강법 기반 옵티마이저는 지역 최적값으로의 수렴 가능성과 해의 저장 및 비교구조가 없다는 단점이 있다. 본 연구에서는 인공신경망을 이용한 수질 예측성능을 향상시키기 위해 개량형 옵티마이저를 개발하여 경사하강법 기반 옵티마이저의 단점을 개선하였다. 본 연구에서 제안한 옵티마이저는 경사하강법 기반 옵티마이저 중 학습오차가 낮은 Adaptive moments (Adam)과 Nesterov-accelerated adaptive moments (Nadam)를 Harmony Search (HS) 또는 Novel Self-adaptive Harmony Search (NSHS)와 결합한 옵티마이저이다. 개량형 옵티마이저의 학습 및 예측성능 평가를 위해 개량형 옵티마이저를 Long Short-Term Memory (LSTM)에 적용하여 국내의 다산 수질관측소의 수질인자인 수온, 용존산소량, 수소이온농도 및 엽록소-a를 학습 및 예측하였다. 학습결과를 비교하면, Nadam combined with NSHS (NadamNSHS)를 사용한 LSTM의 Mean Squared Error (MSE)가 0.002921로 가장 낮았다. 또한, 각 옵티마이저별 4개 수질인자에 대한 MSE 및 R²에 따른 예측순위를 비교하였다. 각 옵티마이저의 평균 순위를 비교하면, NadamNSHS를 사용한 LSTM이 2.25로 가장 높은 것을 확인하였다.

키워드

수질인자

Long Short-Term Memory

Novel Self-adaptive Harmony Search

개량형 옵티마이저

References

Akkoyunlu, A., and Akiner, M.E. (2010). "Feasibility assessment of data-driven models in predicting pollution trends of Omerli Lake, Turkey." Water Resources Management, Vol. 24, No. 13, pp. 3419-3436. 10.1007/s11269-010-9613-0

Babu, C.N., and Reddy, B.E. (2014). "A moving-average filter based hybrid ARIMA-ANN model for forecasting time series data." Applied Soft Computing, Vol. 23, pp. 27-38. 10.1016/j.asoc.2014.05.028

Cai, Q., Zhang, D., Zheng, W., and Leung, S.C. (2015). "A new fuzzy time series forecasting model combined with ant colony optimization and auto-regression." Knowledge-Based Systems, Vol. 74, pp. 61-68. 10.1016/j.knosys.2014.11.003

Cao, X., Liu, H., and Chen, N. (1997). "Classification of Cm I energy levels using PCA-BPN and PCA-NLM." Chemical Physics, Vol. 220, No. 3, pp. 289-297. 10.1016/S0301-0104(97)00139-0

Chen, Y., Song, L., Liu, Y., Yang, L., and Li, D. (2020). "A review of the artificial neural network models for water quality prediction." Applied Sciences, Vol. 10, No. 17, 5776. 10.3390/app10175776

Choi, Y.H., Eghdami, S., Ngo, T.T., Chaurasia, S.N., and Kim, J.H. (2019). "Comparison of parameter-setting-free and self-adaptive harmony search." Proceedings Harmony Search and Nature Inspired Optimization Algorithms, Singapore, pp. 105-112. 10.1007/978-981-13-0761-4_11

Dogan, E., Ates, A., Yilmaz, C., and Eren, B. (2008). "Application of artificial neural networks to estimate wastewater treatment plant inlet biochemical oxygen demand." Environmental Progress, Vol. 27, No. 4, pp.439-446. 10.1002/ep.10295

Faruk, D.Ö. (2010). "A hybrid neural network and ARIMA model for water quality time series prediction." Engineering Applications of Artificial Intelligence, Vol. 23, No. 4, pp. 586-594. 10.1016/j.engappai.2009.09.015

Folorunso, T.A., Aibinu, A.M., Kolo, J.G., Sadiku, S.O., and Orire, A.M. (2018). "Effects of data normalization on water quality model in a recirculatory aquaculture system using artificial neural network." I-manager's Journal on Pattern Recognition, Vol. 5, No. 3, 21. 10.26634/jpr.5.3.15678

Geem, Z.W. (2006). "Optimal cost design of water distribution networks using harmony search." Engineering Optimization, Vol. 38, No. 3, pp. 259-277. 10.1080/03052150500467430

Geem, Z.W., Kim, J.H., and Loganathan, G.V. (2001). "A new heuristic optimization algorithm: Harmony search." Simulation, Vol. 76, No. 2, pp. 60-68. 10.1177/003754970107600201

Hasan, H., and Tahir, N.M. (2010). "Feature selection of breast cancer based on principal component analysis." In 2010 6th International Colloquium on Signal Processing & its Applications, Malacca, Malaysia, pp. 1-4. 10.1109/CSPA.2010.5545298

Hochreiter, S., and Schmidhuber, J. (1997). "Long short-term memory." Neural Computation, Vol. 9, No. 8, pp. 1735-1780. 10.1162/neco.1997.9.8.17359377276

Jin, L., Kuang, X., Huang, H., Qin, Z., and Wang, Y. (2005). "Study on the overfitting of the artificial neural network forecasting model." Acta Meteorologica Sinica, Vol. 19, No. 2, 216.

Joo, D.S., Choi, D.J., and Park, H. (2000). "The effects of data preprocessing in the determination of coagulant dosing rate." Water Research, Vol. 34, No. 13, pp. 3295-3302. 10.1016/S0043-1354(00)00067-1

Joo, G., Park, C., and Im, H. (2020). "Performance evaluation of machine learning optimizers." Journal of Institute of Korean Electrical and Engineers, Vol. 24, No, 3, pp. 766-776.

Khatri, P., Gupta, K.K., and Gupta, R.K. (2021). "Drift compensation of commercial water quality sensors using machine learning to extend the calibration lifetime." Journal of Ambient Intelligence and Humanized Computing, Vol. 12, pp. 3091-3099. 10.1007/s12652-020-02469-y

Lee, W.J., and Lee, E.H. (2022a). "Runoff prediction based on the discharge of pump stations in an urban stream using a modified multi-layer perceptron combined with meta-heuristic optimization." Water, Vol. 14, No. 1, 99. 10.3390/w14010099

Lee, W.J., and Lee, E.H. (2022b). "Improvement of multi layer perceptron performance using combination of gradient descent and harmony search for prediction of ground water level." Journal of Korea Water Resources Association, Vol. 55, No. 11, pp. 903-911.

Li, L., Jiang, P., Xu, H., Lin, G., Guo, D., and Wu, H. (2019). "Water quality prediction based on recurrent neural network and improved evidence theory: a case study of Qiantang River, China." Environmental Science and Pollution Research, Vol. 26, No. 19, pp. 19879-19896. 10.1007/s11356-019-05116-y31093910

Lu, H., and Ma, X. (2020). "Hybrid decision tree-based machine learning models for short-term water quality prediction." Chemosphere, Vol. 249, 126169. 10.1016/j.chemosphere.2020.12616932078849

Luo, K. (2013). "A novel self-adaptive harmony search algorithm." Journal of Applied Mathematics, Vol. 2013, pp. 1-16. 10.1155/2013/653749

Manjarres, D., Landa-Torres, I., Gil-Lopez, S., Del Ser, J., Bilbao, M.N., Salcedo-Sanz, S., and Geem, Z.W. (2013). "A survey on applications of the harmony search algorithm." Engineering Applications of Artificial Intelligence, Vol. 26, No. 8, pp. 1818-1831. 10.1016/j.engappai.2013.05.008

McCulloch, W.S., and Pitts, W. (1943). "A logical calculus of the ideas immanent in nervous activity." The Bulletin of Mathematical Biophysics, Vol. 5, No. 4, pp. 115-133. 10.1007/BF02478259

Mok, J.Y., Choi, J.H., and Moon, Y.I. (2020). "Prediction of multipurpose dam inflow using deep learning." Journal of Korea Water Resources Association, Vol. 53, No. 2, pp. 97-105.

Nawi, N.M., Atomi, W.H., and Rehman, M.Z. (2013). "The effect of data pre-processing on optimized training of artificial neural networks." Procedia Technology, Vol. 11, pp. 32-39. 10.1016/j.protcy.2013.12.159

Pan, Q.K., Suganthan, P.N., Tasgetiren, M.F., and Liang, J.J. (2010). "A self-adaptive global best harmony search algorithm for continuous optimization problems." Applied Mathematics and Computation, Vol. 216, pp. 830-848. 10.1016/j.amc.2010.01.088

Park, S.Y., Choi, J.H., Wang, S., and Park, S.S. (2006). "Design of a water quality monitoring network in a large river system using the genetic algorithm." Ecological modelling, Vol. 199, No. 3, pp. 289-297. 10.1016/j.ecolmodel.2006.06.002

Rosenblatt, F. (1958). "The perceptron: A probabilistic model for information storage and organization in the brain." Psychological Review, Vol. 65, No. 6, 386. 10.1037/h004251913602029

Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986). "Learning representations by back-propagating errors." Nature, Vol. 323, No. 6088, pp. 533-536. 10.1038/323533a0

Ryu, Y.M., and Lee, E.H. (2022). "Application of neural networks to predict Daecheong Dam water levels." Journal of the Korean Society of Hazard Mitigation, Vol. 22, No. 1, pp. 67-78. 10.9798/KOSHAM.2022.22.1.67

Sedki, A., Ouazar, D., and El Mazoudi, E. (2009). "Evolving neural network using real coded genetic algorithm for daily rainfall-runoff forecasting." Expert Systems with Applications, Vol. 36, No. 3, pp. 4523-4527. 10.1016/j.eswa.2008.05.024

Wang, T.S., Tan, C.H., Chen, L., and Tsai, Y.C. (2008). "Applying artificial neural networks and remote sensing to estimate chlorophyll-a concentration in water body." Proceedings of the 2008 2nd International Symposium Intelligent Information Technology Application IITA, Shanghai, China, pp. 540-544. 10.1109/IITA.2008.279

Wang, Y., Zhou, J., Chen, K., Wang, Y., and Liu, L. (2017). "Water quality prediction method based on LSTM neural network." Proceedings 2017 12th International Conference on Intelligent Systems and Knowledge Engineering, Nanjing, China, pp. 1-5. 10.1109/ISKE.2017.8258814

Xiang, Y., and Jiang, L. (2009). "Water quality prediction using LS-SVM and particle swarm optimization." Proceedings 2009 Second International Workshop on Knowledge Discovery and Data Mining, Moscow, Russia, pp. 900-904. 10.1109/WKDD.2009.217

Zare, A., Bayat, V., and Daneshkare, A. (2011). "Forecasting nitrate concentration in groundwater using artificial neural network and linear regression models." International Agrophysics, Vol. 25, No. 2, pp. 187-192.

Zhang, L., Zou, Z., and Shan, W. (2017). "Development of a method for comprehensive water quality forecasting and its application in Miyun reservoir of Beijing, China." Journal of Environmental Sciences, Vol. 56, pp. 240-246. 10.1016/j.jes.2016.07.01728571859

Zheng, L., Wang, H., Liu, C., Zhang, S., Ding, A., Xie, E., Li, J., and Wang, S. (2021). "Prediction of harmful algal blooms in large water bodies using the combined EFDC and LSTM models." Journal of Environmental Management, Vol. 295, 113060. 10.1016/j.jenvman.2021.11306034167054

Information

Publisher :KOREA WATER RESOURECES ASSOCIATION
Publisher(Ko) :한국수자원학회
Journal Title :Journal of Korea Water Resources Association
Journal Title(Ko) :한국수자원학회 논문집
Volume : 57
No :2
Pages :73-85
Received Date : 2023-12-19
Revised Date : 2024-01-08
Accepted Date : 2024-01-22
DOI :https://doi.org/10.3741/JKWRA.2024.57.2.73

Journal of Korea Water Resources Association ISSN:2799-8746(Print) 2799-8754(Online) 한국수자원학회 논문집

All Issue