Comparative assessment of frost event prediction models using logistic regression,  random forest,  and LSTM networks

Jong Ahn Chun; Hyun-Ju Lee; Seul-Hee Im; Daeha Kim; Sang-Soo Baek

doi:10.3741/JKWRA.2021.54.9.667

All Issue

2021 Vol.54, Issue 9 Preview Page Next Page

Research Article

Comparative assessment of frost event prediction models using logistic regression, random forest, and LSTM networks 로지스틱 회귀, 랜덤포레스트, LSTM 기법을 활용한 서리예측모형 평가

30 September 2021. pp. 667-680

PDF XML

Abstract

We investigated changes in frost days and frost-free periods and to comparatively assess frost event prediction models developed using logistic regression (LR), random forest (RF), and long short-term memory (LSTM) networks. The meteorological variables for the model development were collected from the Suwon, Cheongju, and Gwangju stations for the period of 1973-2019 for spring (March - May) and fall (September - November). The developed models were then evaluated by Precision, Recall, and f-1 score and graphical evaluation methods such as AUC and reliability diagram. The results showed that significant decreases (significance level of 0.01) in the frequencies of frost days were at the three stations in both spring and fall. Overall, the evaluation metrics showed that the performance of RF was highest, while that of LSTM was lowest. Despite higher AUC values (above 0.9) were found at the three stations, reliability diagrams showed inconsistent reliability. A further study is suggested on the improvement of the predictability of both frost events and the first and last frost days by the frost event prediction models and reliability of the models. It would be beneficial to replicate this study at more stations in other regions.

Keywords

Frost

Frost-free period

Logistic regression

Random forest

Long short-term memory (LSTM) network

이 연구의 목적은 서리 발생일과 무상일 기간의 특성을 분석하고 로지스틱 회귀, 랜덤 포레스트, Long-short Term Memory (LSTM) 기법을 활용하여 서리발생 예측모델을 개발하고 평가하는데 있다. 수원, 청주, 광주 지점에서 봄철과 가을철 서리발생 예측모델 개발을 위한 기상변수들을 수집하였으며, 수집기간은 1973년부터 2019년까지이다. 프리시전(precision), 리콜(Recall), f-1 스코어와, AUC 및 Reliability Diagram과 같은 그래피컬 평가기법을 이용해 서리발생 예측모델을 평가하였다. 봄철과 가을철 모두 서리발생일이 줄어드는 경향성(유의수준: 0.01)을 보였다. 0.9 이상의 높은 AUC 값에도 불구하고, 신뢰도는 일정한 값을 보여주지는 않았다. 서리발생일 측뿐만 아니라, 초상일과 종상일을 정확히 예측할 수 있도록 모형 개선이 필요해 보이며, 다른 지역의 더 많은 지점에서 동일한 기법을 적용해 보는 연구가 필요해 보인다.

키워드

서리

무상일

로지스틱 회귀

랜덤 포레스트

LSTM

References

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jozefowicz, R., Jia, Y., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems, accessed 4 September 2021, <https://tensorflow.org/>.

Bonsal, B.R., Zhang, X., Vincent, L.A., and Hogg, W.D. (2001). "Characteristics of daily and extreme temperatures over Canada." Journal of Climate, Vol. 14, pp. 1959-1976. 10.1175/1520-0442(2001)014<1959:CODAET>2.0.CO;2

Breiman, L. (2001). "Random forests." Machine Learning, Vol. 45, No. 1, pp. 5-32. 10.1023/A:1010933404324

Chevalier, R.F., Hoogenboom, G., McClendon, R.W., and Paz, J.O. (2012). "A web-based fuzzy expert system for frost warnings in horticultural crops." Environmental Modelling and Software, Vol. 35, pp. 84-91. 10.1016/j.envsoft.2012.02.010

Ding, L., Noborio, K., and Shibuya, K. (2019). "Frost forecast using machine learning-from association to causality." Procedia Computer Science, Vol. 159, pp. 1001-1010. 10.1016/j.procs.2019.09.267

Easterling, D.R. (2002). "Recent changes in frost days and the frost-free season in the United States." Bulletin of the American Meteorological Society, Vol. 83, No. 9, pp. 1327-1332. 10.1175/1520-0477-83.9.1327

Frich, P., Alexander, L.V., Della-Marta, P., Gleason, B., Haylock, M., Tank, A.K., and Peterson, T. (2002). "Global changes in climatic extremes during the 2nd half of the 20th century." Climate Research, Vol. 19, pp. 193-212. 10.3354/cr019193

Han, J. H., Choi., J.J., Chung, U., Cho, K.S., and Chun, J.P. (2009). "Frostfall forecasting in the Naju pear production area based on discriminant analysis of climatic data." Korean Journal of Agricultural and Forest Meteorology, Vol. 11, No. 4, pp. 135-142. (in Korean with English abstract) 10.5532/KJAFM.2009.11.4.135

Heino, R., Brazdil, R., Forland, E., Tuomenvirta, H., Alexandersson, H., Beniston, M., Pfister, C., Rebetez, M., Rosenhagen, G., Rosner, S., and Wigib, J. (1999). "Progress in the study of climate extremes in northern and central Europe." Climatic Change, Vol. 42, pp. 151-181. 10.1023/A:1005420400462

Hochreiter, S., and Schmidhuber, J. (1997). "Long short-term memory." Neural Computation, Vol. 9, No. 8, pp. 1735-1780. 10.1162/neco.1997.9.8.17359377276

Hosmer Jr, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied logistic regression. John Wiley & Sons. Inc., Hoboken, NJ, U.S., p. 528. 10.1002/9781118548387

Kalma, J.D., Laughlin, G.P., Caprio, J.M., and Hamer, P.J.C. (1992). The bioclimatology of frost: Its occurrence, impact and protection. Springer, Berlin, Germany, p.144. 10.1007/978-3-642-58132-8

Kim, Y., Shim, K.-M., Jung, M.-P., and Choi, I.-T. (2017). "Study on the estimation of frost occurrence classification using machine learning methods." Korean Journal of Agricultural and Forest Meteorology, Vol. 19, No. 3, pp. 86-92. (in Korean with English abstract)

Kleinbaum, D.G., Dietz, K., Gail, M., Klein, M., and Klein, M. (2002). Logistic regression. Springer, New York, NY, U.S., p. 702.

Kwon, Y.-A., Lee, H.-S. Kown, W.-T., and Boo, K.-O. (2008). "The weather characteristics of frost occurrence days for protecting crops against frost damage." Journal of the Korean Geographical Society, Vol. 43, No. 6, pp. 824-842. (in Korean with English abstract)

Lee, H., Chun, J.A,. Han, H.-H., and Kim, S. (2016). "Prediction of frost occurrences using statistical modeling approaches." Advances in Meteorology, Vol. 2016, Article ID 2075186. 10.1155/2016/2075186

Lee, S. (2005). "Application of logistic regression model and its validation for landslide susceptibility mapping using GIS and remote sensing data." International Journal of Remote Sensing, Vol. 26, No. 7, pp. 1477-1491. 10.1080/01431160412331331012

Naghibi, S.A., Pourghasemi, H.R., and Dixon, B. (2016). "GIS-based groundwater potential mapping using boosted regression tree, classification and regression tree, and random forest machine learning models in Iran." Environmental Monitoring and Assessment, Vol. 188, No. 1, p. 44. 10.1007/s10661-015-5049-626687087

Ozdemir, A. (2011). "Using a binary logistic regression method and GIS for evaluating and mapping the groundwater spring potential in the Sultan Mountains (Aksehir, Turkey)." Journal of Hydrology, Vol. 405, No. 1-2, pp. 123-136. 10.1016/j.jhydrol.2011.05.015

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). "Scikit-learn: Machine learning in Python." Journal of Machine Learning Research, Vol. 12, pp. 2825-2830.

Rozante, J.R., Gutierrez, E.R., da Silva Dias, P.L., de Almeida Fernandes, A., Alvim, D.S., and Silva, V.M. (2020). "Development of an index for frost prediction: Technique and validation." Meteorological Applications, Vol. 27, No. 1, pp. 1-12. 10.1002/met.1807

Sgubin, G., Swingedouw, D., Dayon, G., de Cortazar-Atauri, I.G., Ollat, N., Page, C., and van Leeuwen, C. (2018). "The risk of tardive frost damage in French vineyards in a changing climate." Agricultural and Forest Meteorology, Vol. 250-251, pp. 226-242. 10.1016/j.agrformet.2017.12.253

Snyder, R.L., Paw, U.K.T., and Thompson, J.F. (1987). Passive frost protection of trees and vines, accessed 4 September 2121, <https://anrcatalog.ucanr.edu/pdf/21429e.pdf>.

Temeyer, B.R., Gallus Jr., W.A., Jungbluth, K.A., Burkheimer, D., and McCauley, D. (2003). "Using an Artificial neural network to predict parameters for frost deposition on Iowa Bridgeways." Proceedings of the 2003 Mid-Continent Transportation Research Symposium. Iowa State University, August 21-22, 2003, Ames, IA. accessed 4 September 2021, <http://www.ctre.iastate.edu/pubs/midcon2003/TemeyerFrost.pdf>.

Vitasse, Y., Schneider, L., Rixen, C., Christen, D., and Rebetez, M. (2018). "Increase in the risk of exposure of forest and fruit trees to spring frosts at higher elevations in Switzerland over the last four decades." Agricultural and Forest Meteorology, Vol. 248, pp. 60-69. 10.1016/j.agrformet.2017.09.005

Xiao, L., Liu, L., Asseng, S., Xia, Y., Tang, L, Liu, B., Cao, W., and Zhu, Y. (2018). "Estimating spring frost and its impact on yield across winter wheat in China." Agricultural and Forest Meteorology, Vol. 260-261, pp. 154-164. 10.1016/j.agrformet.2018.06.006

Xue, D., De Baets, B., Van Cleemput, O., Hennessy, C., Berglund, M., and Boeckx, P. (2013). "Classification of nitrate polluting activities through clustering of isotope mixing model outputs." Journal of Environmental Quality, Vol. 42, No. 5, pp. 1486-1497. 10.2134/jeq2012.045624216426

Information

Publisher :KOREA WATER RESOURECES ASSOCIATION
Publisher(Ko) :한국수자원학회
Journal Title :Journal of Korea Water Resources Association
Journal Title(Ko) :한국수자원학회 논문집
Volume : 54
No :9
Pages :667-680
Received Date : 2021-06-08
Revised Date : 2021-07-04
Accepted Date : 2021-07-04
DOI :https://doi.org/10.3741/JKWRA.2021.54.9.667

Journal of Korea Water Resources Association ISSN:2799-8746(Print) 2799-8754(Online) 한국수자원학회 논문집

All Issue