Prediction of dam inflow based on LSTM-s2s model using luong attention

Jonghyeok Lee; Suyeon Choi; Yeonjoo Kim

doi:10.3741/JKWRA.2022.55.7.495

All Issue

2022 Vol.55, Issue 7 Preview Page Next Page

Research Article

Prediction of dam inflow based on LSTM-s2s model using luong attention Attention 기법을 적용한 LSTM-s2s 모델 기반 댐유입량 예측 연구

31 July 2022. pp. 495-504

PDF XML

Abstract

With the recent development of artificial intelligence, a Long Short-Term Memory (LSTM) model that is efficient with time-series analysis is being used to increase the accuracy of predicting the inflow of dams. In this study, we predict the inflow of the Soyang River dam, using the LSTM model with the Sequence-to-Sequence (LSTM-s2s) and attention mechanism (LSTM-s2s with attention) that can further improve the LSTM performance. Hourly inflow, temperature, and precipitation data from 2013 to 2020 were used to train the model, and validate and test for evaluating the performance of the models. As a result, the LSTM-s2s with attention showed better performance than the LSTM-s2s in general as well as in predicting a peak value. Both models captured the inflow pattern during the peaks but detailed hourly variability is limitedly simulated. We conclude that the proposed LSTM-s2s with attention can improve inflow forecasting despite its limits in hourly prediction.

Keywords

LSTM

Sequence-to-Sequence

Attention

Dam Inflow prediction

최근 인공지능의 발전으로 시계열 자료 분석에 효과적인 Long Short-Term Memory (LSTM) 모델이 댐유입량 예측의 정확도를 높이는 데 활용되고 있다. 본 연구에서는 그 중 LSTM의 성능을 더욱 향상할 수 있는 Sequence-to-Sequence (s2s) 구조에 Attention 기법을 LSTM 모델에 첨가하여 소양강댐 유역의 유입량을 예측하였다. 분석 데이터는 2013년부터 2020년까지의 유입량 시자료와 종관기상관측기온 및 강수량 자료를 학습, 검증, 평가로 나누어 훈련한 후, 모델의 성능 평가를 진행하였다. 분석 결과, LSTM-s2s 모델보다 attention까지 첨가한 모델이 일반적으로 더 좋은 성능을 보였으며, attention 첨가 모델이 첨두값도 더 잘 예측하는 모습을 보였다. 그리고 두 모델 모두 첨두값 발생 동안 유량 패턴을 잘 반영하였지만 세밀한 시간 단위 변화량에는 어려움이 있었다. 이를 통해 시간 단위 예측의 어려움에도 불구하고, LSTM-s2s에 attention까지 첨가한 모델이 기존 LSTM-s2s의 예측 성능을 향상할 수 있음을 알 수 있었다.

키워드

LSTM

Sequence-to-sequence

Attention

댐유입량 예측

References

Bicknell, B.R., Imhoff, J.C., Kittle Jr, J.L., Donigian Jr, A.S., and Johanson, R.C. (1996). Hydrological simulation program-FORTRAN. User's Manual for Release 11. U.S. Environmental Protection Agency, Washington, D.C., U.S.

Cho, K., and Kim, Y. (2022). "Improving streamflow prediction in the WRF-Hydro model with LSTM networks." Journal of Hydrology, Vol. 605, 127297. 10.1016/j.jhydrol.2021.127297

Choi, H., Cho, K., and Bengio, Y. (2018). "Fine-grained attention mechanism for neural machine translation." Neurocomputing, Vol. 284, pp. 171-176. 10.1016/j.neucom.2018.01.007

Choi, J.-H., Kim, J.-S., Gwon, J.-H., and Moon, Y.-I. (2016). "Weighting assessment on evaluation indicators of dam rehabilitation using the AHP analysis." Journal of Korea Water Resources Association, Vol. 49, No. 5, pp. 381-389. 10.3741/JKWRA.2016.49.5.381

Costa-jussà, M.R. (2018). "From feature to paradigm: Deep learning in machine translation." The Journal of Artificial Intelligence Research, Vol. 61, pp. 947-974. 10.1613/jair.1.11198

Dawson, C.W., and Wilby, R.L. (2001). "Hydrological modelling using artificial neural networks." Progress in Physical Geography, Vol. 25, No. 1, pp. 80-108.

Devia, G.K., Ganasri, B.P., and Dwarakish, G.S. (2015). "A review on hydrological models." Aquatic Procedia, Vol. 4, pp. 1001-1007. 10.1016/j.aqpro.2015.02.126

Fu, M., Fan, T., Ding, Z., Salih, S.Q., Al-Ansari, N., and Yaseen, Z.M. (2020). "Deep learning data-intelligence model based on adjusted forecasting window scale: Application in daily streamflow simulation." IEEE Access, Vol. 8, pp. 32632-32651. 10.1109/ACCESS.2020.2974406

Han, H., Choi, C., Jung, J., and Kim, H.S. (2021). "Application of sequence to sequence learning based LSTM model (LSTM-s2s) for forecasting dam inflow." Journal of Korea Water Resources Association, Vol. 54, No. 3, pp. 157-166. 10.3741/JKWRA.2021.54.3.157

He, X., Luo, J., Li, P., Zuo, G., and Xie, J. (2020). "A hybrid model based on variational mode decomposition and gradient boosting regression tree for monthly runoff forecasting." Water Resources Management, Vol. 34, No. 2, pp. 865-884. 10.1007/s11269-020-02483-x

Hochreiter, S., and Schmidhuber, J. (1997). "Long short-term memory." Neural Computation, Vol. 9, No. 8, pp. 1735-1780. 10.1162/neco.1997.9.8.17359377276

Jung, S., Lee, D., and Lee, K. (2017). "Prediction of river water level using deep-learning open library." Korean Society of Hazard Mitigation, Vol. 18, No. 1, pp. 1-11. 10.9798/KOSHAM.2018.18.1.1

Kao, I.-F., Zhou, Y., Chang, L.-C., and Chang, F.-J. (2020). "Exploring a long short-term memory based encoder-decoder framework for multi-step-ahead flood forecasting." Journal of Hydrology, Vol. 583, 124631. 10.1016/j.jhydrol.2020.124631

Kim, B.K., Kim, S., Lee, E.T., and Kim, H.S. (2007). "Methodology for estimating ranges of SWAT model parameters: Application to Imha Lake inflow and suspended sediments." Journal of The Korean Society of Civil Engineers B, Vol. 27 No. 6B, pp. 661-668.

Kwak, J.-W., Kim, H.-S., and Hong, I.-P. (2009). "A study of progressive parameter calibrations for rainfall-runoff models." Proceedings of the Korea Water Resources Association Conference, pp. 1499-1503.

Kwon, O.-I., and Shim, M.-P. (1997). "Determination scheme of variable restricted water level during flood period of multipurpose dam." Journal of Korea Water Resources Association, Vol. 30, No. 6, pp. 709-720.

Le, X.-H., Ho, H. V., Lee, G., and Jung, S. (2019). "Application of Long Short-Term Memory (LSTM) neural network for flood forecasting." Water, Vol. 11, No. 7, 1387. 10.3390/w11071387

Legates, D.R., and McCabe Jr., G.J. (1999). "Evaluating the use of "goodness-of-fit" Measures in hydrologic and hydroclimatic model validation." Water Resources Research, Vol. 35, No. 1, pp. 233-241. 10.1029/1998WR900018

Li, W., Guo, D., and Fang, X. (2018). "Multimodal architecture for video captioning with memory networks and an attention mechanism." Pattern Recognition Letters, Vol. 105, pp. 23-29. 10.1016/j.patrec.2017.10.012

Li, Z., Liu, W., Zhang, X., and Zheng, F. (2009). "Impacts of land use change and climate variability on hydrology in an agricultural catchment on the loess plateau of China." Journal of Hydrology, Vol. 377, No. 1-2, pp. 35-42. 10.1016/j.jhydrol.2009.08.007

Luong, M.-T., Pham, H., and Manning, C.D. (2015). "Effective approaches to attention-based neural machine translation." Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, ACL, Lisbon, Portugal, pp. 1412-1421. 10.18653/v1/D15-1166

Marmanis, D., Schindler, K., Wegner, J. D., Galliani, S., Datcu, M., and Stilla, U. (2018). "Classification with an edge: Improving semantic image segmentation with boundary detection." ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 135, pp. 158-172. 10.1016/j.isprsjprs.2017.11.009

National Water Resources Management Infomation System (WAMIS) (2003). South Korea, accessed 10, January 2022, <www.wamis.go.kr>.

Noh, J.-K., and Lee, J.-N. (2011). "Comparison of streamflow runoff model in Korea for applying to reservoir operation." Korean Journal of Agricultural Science, Vol. 38, No. 3, pp. 513-524.

Nourani, V. (2017). "An emotional ANN (EANN) approach to modeling rainfall-runoff process." Journal of Hydrology, Vol. 544, pp. 267-277. 10.1016/j.jhydrol.2016.11.033

Nourani, V., Komasi, M., and Mano, A. (2009). "A multivariate ANN-wavelet approach for rainfall-runoff modeling." Water Resources Management, Vol. 23, No. 14, pp. 2877-2894. 10.1007/s11269-009-9414-5

Qi, Y., Zhou, Z., Yang, L., Quan, Y., and Miao, Q. (2019). "A decomposition-ensemble learning model based on LSTM neural network for daily reservoir inflow forecasting." Water Resources Management, Vol. 33, No. 12, pp. 4123-4139. 10.1007/s11269-019-02345-1

Song, K., Yao, T., Ling, Q., and Mei, T. (2018). "Boosting image sentiment analysis with visual attention." Neurocomputing, Vol. 312, pp. 218-228. 10.1016/j.neucom.2018.05.104

Sutskever, I., Vinyals, O., and Le, Q.V. (2014). "Sequence to sequence learning with neural networks." Proceedings of the 27th International Conference on Neural Information Processing Systems, MIT Press, Cambridge, MA, U.S., Vol. 2, pp. 3104-3112.

Tiwari, M.K., and Chatterjee, C. (2010). "Development of an accurate and reliable hourly flood forecasting model using wavelet-bootstrap-ANN (WBANN) hybrid approach." Journal of Hydrology, Vol. 394, No. 3-4, pp. 458-470. 10.1016/j.jhydrol.2010.10.001

Xiang, Z., Yan, J., and Demir, I. (2020). "A rainfall-runoff model with LSTM-based sequence-to-sequence learning." Water Resources Research, Vol. 56, No. 1, e2019WR025326. 10.1029/2019WR025326

Yan, L., Chen, C., Hang, T., and Hu, Y. (2021). "A stream prediction model based on attention-LSTM." Earth Science Informatics, Vol. 14, No. 2, pp. 723-733. 10.1007/s12145-021-00571-z

Zhou, H., Zhang, Y., Yang, L., Liu, Q., Yan, K., and Du, Y. (2019). "Short-term photovoltaic power forecasting based on long short term memory neural network and attention mechanism." IEEE Access, Vol. 7, pp. 78063-78074. 10.1109/ACCESS.2019.2923006

Information

Publisher :KOREA WATER RESOURECES ASSOCIATION
Publisher(Ko) :한국수자원학회
Journal Title :Journal of Korea Water Resources Association
Journal Title(Ko) :한국수자원학회 논문집
Volume : 55
No :7
Pages :495-504
Received Date : 2022-04-12
Revised Date : 2022-05-27
Accepted Date : 2022-05-30
DOI :https://doi.org/10.3741/JKWRA.2022.55.7.495

Journal of Korea Water Resources Association ISSN:2799-8746(Print) 2799-8754(Online) 한국수자원학회 논문집

All Issue