Developing a regional fog prediction model using tree-based machine-learning techniques and automated visibility observations

Daeha Kim

doi:10.3741/JKWRA.2021.54.12.1255

All Issue

2021 Vol.54, Issue 12 Preview Page Next Page

Research Article

Developing a regional fog prediction model using tree-based machine-learning techniques and automated visibility observations 시정계 자료와 기계학습 기법을 이용한 지역 안개예측 모형 개발

31 December 2021. pp. 1255-1263

PDF XML

Abstract

While it could become an alternative water resource, fog could undermine traffic safety and operational performance of infrastructures. To reduce such adverse impacts, it is necessary to have spatially continuous fog risk information. In this work, tree-based machine-learning models were developed in order to quantify fog risks with routine meteorological observations alone. The Extreme Gradient Boosting (XGB), Light Gradient Boosting (LGB), and Random Forests (RF) were chosen for the regional fog models using operational weather and visibility observations within the Jeollabuk-do province. Results showed that RF seemed to show the most robust performance to categorize between fog and non-fog situations during the training and evaluation period of 2017-2019. While the LGB performed better than in predicting fog occurrences than the others, its false alarm ratio was the highest (0.695) among the three models. The predictability of the three models considerably declined when applying them for an independent period of 2020, potentially due to the distinctively enhanced air quality in the year under the global lockdown. Nonetheless, even in 2020, the three models were all able to produce fog risk information consistent with the spatial variation of observed fog occurrences. This work suggests that the tree-based machine learning models could be used as tools to find locations with relatively high fog risks.

Keywords

Fog occurrence

Tree-based machine-learning

Regional prediction

안개는 대체수자원이 될 수 있으나 교통사고 위험을 높이고 공항 운영에 제약을 가하는 사회적 영향이 큰 기상현상이다. 본 연구에서는 1 km 미만 가시거리(시정)로 정의되는 안개 발생을 기상자료로 예측하는 지역 기계학습모형을 개발하고 그 예측력을 평가하였다. 전라북도 지역의 10개 기상청 지상관측소의 2017-2019년 시정 및 기상관측자료로 앙상블 분류기법인 Extreme Gradient Boosting (XGB), Light Gradient Boosting (LGB), Random Forests (RF)를 학습시켜 지역 안개 모형을 개발하였고 독립적인 2020년 자료로 모형의 사용성을 평가하였다. 그 결과, 학습·검증기간(2017-2019)에는 True Skill Score를 기준으로 가장 높은 예측력을 보인 방법은 LGB 기법이었지만 다른 두 모형에 비해 False Alarm Ratio가 컸다. RF 모형과 XGB 방법 역시 기존 연구에 상응하는 예측성능을 보이는 것으로 확인되었다. 2020년 자료를 입력해 안개 발생을 모의했을 때 세 모형의 예측성능은 2017-2019년 기간보다 떨어졌지만 모두 관측 안개일수의 공간분포와 일관되는 안개 위험을 예측했다. 세 기계학습 모형은 안개위험이 상대적으로 높은 지역을 추출하는 기법으로 사용이 가능할 것으로 보인다.

키워드

안개 발생예측

의사결정나무 기반 기계학습

지역예측모형

References

Bari, D., and Bergot, T. (2018). “Influence of environmental conditions on forecasting of an advection-radiation fog: A case study from the Casablanca region, Morocco.” Aerosol and Air Quality Research, Vol. 18, pp. 62-78. 10.4209/aaqr.2016.11.0520

Bari, D., and Ouagabi, A. (2020). “Machine-learning regression applied to diagnose horizontal visibility from mesoscale NWP model forecasts.” SN Applied Sciences, Vol. 2, No. 4, pp. 1-13. 10.1007/s42452-020-2327-x

Bartoková, I., Bott, A., Bartok, J., and Gera, M. (2015). “Fog prediction for road traffic safety in a coastal desert region: Improvement of nowcasting skills by the machine-learning approach.” Boundary-Layer Meteorology, Vol. 157, No. 3, pp. 501-516. 10.1007/s10546-015-0069-x

Breiman, L. (1996). “Bagging predictors.” Machine Learning, Vol. 24, No. 2, pp. 123-140. 10.1007/BF00058655

Breiman, L. (2001). “Random forests.” Machine Learning, Vol. 45, No. 1, pp. 5-32. 10.1023/A:1010933404324

Breiman, L., Friedman, J.H., Olshen, R.A., and Stone, C.J. (1984). Classification and regression trees. Chapman and Hall/CRC, Boca Raton, FL, U.S.

Chen, T., and Guestrin, C. (2016). “Xgboost: A scalable tree boosting system.” Proceedings of the 22nd acm Sigkdd International Conference on Knowledge Discovery and Data Mining, The Association for Computing Machinery’s Special Interest Group on Knowledge Discovery and Data Mining (SIGKDD), San Francisco, CA, U.S., pp. 785-794. 10.1145/2939672.2939785

Cornejo-Bueno, L., Casanova-Mateo, C., Sanz-Justo, J., Cerro-Prada, E., and Salcedo-Sanz, S. (2017). “Efficient prediction of low-visibility events at airports using machine-learning regression.” Boundary-Layer Meteorology, Vol. 165, No. 2, pp. 349-370. 10.1007/s10546-017-0276-8

Domen, J.K., Stringfellow, W.T., Camarillo, M.K., and Gulati, S. (2014). “Fog water as an alternative and sustainable water resource.” Clean Technologies and Environmental Policy, Vol. 16, No. 2, pp. 235-249. 10.1007/s10098-013-0645-z

Fan, J., Ma, X., Wu, L., Zhang, F., Yu, X., and Zeng, W. (2019). “Light gradient boosting machine: An efficient soft computing model for estimating daily reference evapotranspiration with local and external meteorological data.” Agricultural Water Management, Vol. 225, 105758. 10.1016/j.agwat.2019.105758

Federal Highway Administration (FHWA) (2018). Road weather management program, “How do weather events impacts roads?”, Accessed 4 September 2021, <https://ops.fhwa.dot.gov/weather/ q1_roadimpact.htm>.

Fedorova, N., Levit, V., Da Silva, A.O., and dos Santos, D.M.B. (2013). “Low visibility formation and forecasting on the northern coast of Brazil.” Pure and Applied Geophysics, Vol. 170, No. 4, pp. 689-709. 10.1007/s00024-012-0565-6

Friedman, J., Hastie, T. and Tibshirani, R. (2000). “Additive logistic regression: A statistical view of boosting (with discussion and a rejoinder by the authors).” The Annals of Statistics, Vol. 28, No. 2, pp. 337-407. 10.1214/aos/1016218223

Friedman, J.H. (2002). “Stochastic gradient boosting.” Computational Statistics & Data Analysis, Vol. 38, No. 4, pp. 367-378. 10.1016/S0167-9473(01)00065-2

Gautam, R., and Singh, M.K. (2018). “Urban heat island over Delhi punches holes in widespread fog in the Indo Gangetic Plains.” Geophysical Research Letters, Vol. 45, No. 2, pp. 1114-1121. 10.1002/2017GL076794

Gultepe, I., Tardif, R., Michaelides, S.C., Cermak, J., Bott, A., Bendix, J., Müller, M.D., Pagowski, M., Hansen, B., Ellrod, G., and Jacobs, W. (2007). “Fog research: A review of past achievements and future perspectives.” Pure and Applied Geophysics, Vol. 164, No. 6, pp. 1121-1159. 10.1007/s00024-007-0211-x

Haeffelin, M., Bergot, T., Elias, T., Tardif, R., Carrer, D., Chazette, P., Colomb, M., Drobinski, P., Dupont, E., Dupont, J.C., and Gomes, L. (2010). “PARISFOG: Shedding new light on fog physical processes.” Bulletin of the American Meteorological Society, Vol. 91, No. 6, pp. 767-783. 10.1175/2009BAMS2671.1

Herman, G.R., and Schumacher, R.S. (2016). “Using reforecasts to improve forecasting of fog and visibility for aviation.” Weather and Forecasting, Vol. 31, No. 2, pp. 467-482. 10.1175/WAF-D-15-0108.1

Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., and Simmons, A. (2020). “The ERA5 global reanalysis.” Quarterly Journal of the Royal Meteorological Society, Vol. 146, No. 730, pp. 1999-2049. 10.1002/qj.3803

Jung, M., Reichstein, M., Margolis, H.A., Cescatti, A., Richardson, A.D., Arain, M.A., Arneth, A., Bernhofer, C., Bonal, D., Chen, J., and Gianelle, D. (2011). “Global patterns of land atmosphere fluxes of carbon dioxide, latent heat, and sensible heat derived from eddy covariance, satellite, and meteorological observations.” Journal of Geophysical Research: Biogeosciences, Vol. 116, G00J07. 10.1029/2010JG001566

Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.Y. (2017). “Lightgbm: A highly efficient gradient boosting decision tree.” Advances In Neural Information Processing Systems, Vol. 30, pp. 3146-3154.

Klemm, O., and Lin, N. (2016). “What causes observed fog trends: Air quality or climate change?” Aerosol and Air Quality Research, Vol. 16, No. 5, pp. 1131-1142. 10.4209/aaqr.2015.05.0353

Krzywinski, M., and Altman, N. (2017). “Classification and regression trees.” Nature Methods, Vol. 14, No. 8, pp. 757-758. 10.1038/nmeth.4370

Lee, H.K., and Suh, M.S. (2019). “Objective classification of fog type and analysis of fog characteristics using visibility meter and satellite observation data over South Korea.” Atmosphere Korean Meteorological Society, Vol. 29, No. 5, pp. 639-658.

Liu, F., Wang, M., and Zheng, M. (2021). “Effects of COVID-19 lockdown on global air quality and health.” Science of the Total Environment, Vol. 755, 142533. 10.1016/j.scitotenv.2020.14253333039885PMC7525347

Lorenz, E.N. (1965). “A study of the predictability of a 28-variable atmospheric model.” Tellus, Vol. 17, pp. 321-333. 10.3402/tellusa.v17i3.9076

Marzban, C. (1998). “Scalar measures of performance in rare-event situations.” Weather and Forecasting, Vol. 13, No. 3, pp. 753-763. 10.1175/1520-0434(1998)013<0753:SMOPIR>2.0.CO;2

Miao, K.C., Han, T.T., Yao, Y.Q., Lu, H., Chen, P., Wang, B., and Zhang, J. (2020). “Application of LSTM for short term fog forecasting based on meteorological elements.” Neurocomputing, Vol. 408, pp. 285-291. 10.1016/j.neucom.2019.12.129

Niu, F., Li, Z., Li, C., Lee, K.H., and Wang, M. (2010). “Increase of wintertime fog in China: Potential impacts of weakening of the Eastern Asian monsoon circulation and increasing aerosol loading.” Journal of Geophysical Research: Atmospheres, Vol. 115, D00K20. 10.1029/2009JD013484

Ortega, L., Otero, L.D., and Otero, C., (2019). “Application of machine learning algorithms for visibility classification.” 2019 IEEE International Systems Conference (SysCon), IEEE, Orlando, FL, U.S., pp. 1-5. 10.1109/SYSCON.2019.8836910

Qualls, R.J., and Crago, R.D. (2020). “Graphical interpretation of wet surface evaporation equations.” Water Resources Research, Vol. 56, No. 10, e2019WR026766. 10.1029/2019WR026766

Rebollo, J.J., and Balakrishnan, H. (2014). “Characterization and prediction of air traffic delays.” Transportation research part C: Emerging technologies, Vol. 44, pp. 231-241. 10.1016/j.trc.2014.04.007

Rhee, J., Park, K., Lee, S., Jang, S., and Yoon, S. (2020). “Detecting hydrological droughts in ungauged areas from remotely sensed hydro-meteorological variables using rule-based models.” Natural Hazards, Vol. 103, pp. 2961-2988. 10.1007/s11069-020-04114-5

Seo, J.H., Jeon, H.W., Sung, U.J., and Sohn, J.R. (2020). “Impact of the COVID-19 outbreak on air quality in Korea.” Atmosphere, Vol. 11, No. 10, 1137. 10.3390/atmos11101137

Skamarock, W.C., Klemp, J.B., Dudhia, J., Gill, D.O., Liu, Z., Berner, J., Wang, W., Powers, J.G., Duda, M.G., Barker, D.M., and Huang, X.Y. (2019). A description of the advanced research WRF model version 4. National Center for Atmospheric Research, Boulder, CO, U.S.

Steeneveld, G.J., Ronda, R.J., and Holtslag, A.A.M. (2015). “The challenge of forecasting the onset and development of radiation fog using mesoscale atmospheric models.” Boundary-Layer Meteorology, Vol. 154, No. 2, pp. 265-289. 10.1007/s10546-014-9973-8

Tardif, R., and Rasmussen, R.M. (2007). “Event-based climatology and typology of fog in the New York City region.” Journal of Applied Meteorology and Climatology, Vol. 46, No. 8, pp. 1141-1168. 10.1175/JAM2516.1

Tramontana, G., Jung, M., Schwalm, C.R., Ichii, K., Camps-Valls, G., Ráduly, B., Reichstein, M., Arain, M.A., Cescatti, A., Kiely, G., and Merbold, L. (2016). “Predicting carbon dioxide and energy fluxes across global FLUXNET sites with regression algorithms.” Biogeosciences, Vol. 13, No. 14, pp. 4291-4313. 10.5194/bg-13-4291-2016

Yang, T., Gao, X., Sorooshian, S., and Li, X. (2016). “Simulating California reservoir operation using the classification and regression tree algorithm combined with a shuffled cross validation scheme.” Water Resources Research, Vol. 52, No. 3, pp. 1626-1651. 10.1002/2015WR017394

Zeng, J., Matsunaga, T., Tan, Z.H., Saigusa, N., Shirai, T., Tang, Y., Peng, S., and Fukuda, Y. (2020). “Global terrestrial carbon fluxes of 1999-2019 estimated by upscaling eddy covariance data with a random forest.” Scientific Data, Vol. 7, No. 1, pp. 1-11. 10.1038/s41597-020-00653-532973132PMC7518252

Information

Publisher :KOREA WATER RESOURECES ASSOCIATION
Publisher(Ko) :한국수자원학회
Journal Title :Journal of Korea Water Resources Association
Journal Title(Ko) :한국수자원학회 논문집
Volume : 54
No :12
Pages :1255-1263
Received Date : 2021-09-23
Revised Date : 2021-11-01
Accepted Date : 2021-11-01
DOI :https://doi.org/10.3741/JKWRA.2021.54.12.1255

Journal of Korea Water Resources Association ISSN:2799-8746(Print) 2799-8754(Online) 한국수자원학회 논문집

All Issue