S2S sub-project on verification and products
Highlight: The WWRP/WGNE Joint Working Group on Forecast Verification Research (JWGFVR) is pleased to announce the 7th International Verification Methods Workshop (7IVMW), which will be held in Berlin, Germany, 8-11 May 2017, preceded by a tutorial on forecast verification methods, 3-6 May 2017.
The workshop will have a dedicated session on S2S verification. The S2S community is cordially invited to submit abstracts on S2S verification methodologies to the workshop. Abstract submission is open at the workshop website http://www.7thverificationworkshop.de/
Please note the following deadlines:
- January 31 for tutorial application (open)
- February 27 for abstract submission (open)
- March 31 for registration (to be open soon)
The S2S community is cordially invited to enter the Challenge to Develop and Demonstrate the Best New User-Oriented Forecast Verification Metric launched by the WWRP/WGNE Joint Working Group on Forecast Verification Research (JWGFVR)
The aim of this challenge is to promote user-oriented verification, that is, quantitative assessment of forecast quality in terms that are meaningful to particular forecast users. The scope includes all applications of meteorological and hydrological forecasts. The user-oriented verification metrics contest will help support the WWRP/WCRP projects on High Impact Weather (HIWeather), Subseasonal to Seasonal Prediction (S2S), and Polar Prediction (PPP).
The deadline for entries is 31 October 2016. Click here to find out more, or contact verifchallenge@ucar.edu.
This S2S verification and products wiki page has the following content:
1) Objectives
2) Membership
3) Linkages with coordinated WMO operational activities
3.1) Collaboration between S2S and WMO
3.2) The pilot real-time sub-seasonal MME prediction in WMO LC-LRFMME
4) List of published literature on verification methods of relevance to S2S verification
4.1) Books and technical reports
4.2) Scientific papers
5) List of published literature on S2S verification
5.1) Assessment of S2S systems forecast skill
5.2) Assessment of MJO/ISO forecast skill
5.3) Assessment of monsoon systems forecast skill and associated characteristics
5.4) Applications
5.5) Seamless verification
6) Available reference verification datasets for assessing S2S forecast quality
6.1) Atmospheric parameters (e.g. geopotential height, temperature, SLP, wind, etc)
6.2) Oceanic parameters
6.3) Surface parameters
6.4) Datasets accessible via the KNMI Climate Explorer
7) S2S project models
7.1) Accessing S2S models data
7.2) Visualizing S2S forecast products
1) Objectives
- Recommend verification metrics and datasets for assessing forecast quality of S2S forecasts
- Provide guidance for a potential centralized verification effort for comparing forecast quality of different S2S forecast systems, including the comparison of multi-model and individual ensemble systems and consider linkages with users and applications
The S2S verification science plan provides more detailed information about this sub-project.
2) Membership
Caio Coelho (CPTEC/INPE, Brazil)
Andrew Robertson (IRI, USA)
Richard Graham (UKMO, UK)
Yuhei Takaya (JMA, Japan)
Debra Hudson (BoM, Australia)
Joanne Robbins (UKMO, UK)
Angel Muñoz (GFDL, USA)
3) Linkages with coordinated WMO operational activities
3.1) Collaboration between S2S and WMO
The research performed in S2S has strong linkages with WMO operational activities, particularly with the CBS/CCl Expert Team on Operational Prediction from Sub-seasonal to Longer time-scale (ET-OPSLS) and the WMO Lead Centre for Long Range Forecast Multi-Model Ensemble (LC-LRFMME). The S2S project in collaboration with WMO is therefore bridging research and operation activities to drive science and technology forward for producing better weather/climate information for a number of application sectors.
As part of this collaboration between S2S and WMO the S2S sub-project on verification and products has been conducting the following activities:
- Preparation of questionnaire on subseasonal verification practices in operational centres (both in operations and research) to help identify gaps and guide novel developments. This questionnaire was sent to the 12 designated WMO Global Producing Centres of Long-Range Forecasts (GPCs), the results were discussed with the ET-OPSLS and are summarized in this document.
- Preparation of document on S2S application-oriented activities and operational needs as input for the ET-OPSLS
3.2) The pilot real-time sub-seasonal MME prediction in WMO LC-LRFMME
The WMO Lead Centre for Long Range Forecast Multi-Model Ensemble (LC-LRFMME) has recently developed a pilot system for real-time multi-model subseasonal forecasts using real-time forecasts (and hindcasts) from a subset of models contributing to the WWRP/WCRP S2S research project accessible via ECMWF data archive. Following this link the S2S research community has the opportunity to see the initial developments conducted by the LC-LRFMME and provide feedback for future developments and improvements in this pilot under development system. Subseasonal models from four GPCs are currently used: ECMWF, Exeter, Tokyo and Washington. A range of forecast products has been developed including probabilities for tercile categories of weekly averages of 2m temperature and rainfall as well as the MJO and BSISO indices. Verification has also been generated using ROC curves and scores as well as anomaly pattern correlation for a few case studies.
4) List of published literature on verification methods of relevance to S2S verification
Below in sections 4.1 and 4.2 is a selected list of published literature (including books, technical reports and scientific papers) on verification methodologies of relevance for S2S forecast verification. A more comprehensive list and additional information on forecast verification is available at http://www.cawcr.gov.au/projects/verification/ a website of the WWRP/WGNE Joint Working Group on Forecast Verification Research.
Please note that further down on this wiki page Section 5 provides a list of published literature on S2S verification including in section 5.1 papers on the assessment of S2S systems forecast skill, in section 5.2 papers on the assessment of MJO/ISO forecast skill, in section 5.3 papers on the assessment of monsoon systems forecast skill and associated characteristics, in section 5.4 papers on applications and in section 5.5 on seamless verification.
4.1) Books and technical reports
Jolliffe IT, Stephenson DB (2012) Forecast Verification: A Practitioner's Guide in Atmospheric Science. 2nd Edition. Wiley and Sons Ltd, 274 pp.
Stanski HR, Wilson LJ, Burrows WR (1989) Survey of common verification methods in meteorology. World Weather Watch Tech. Rept. No.8, WMO/TD No.358, WMO, Geneva, 114 pp. Available here.
Wilks DS (2011) Statistical Methods in the Atmospheric Sciences. 3rd Edition. Elsevier, 676 pp.
4.2) Scientific papers
Bradley AA, Hashino T, Schwartz SS (2003) Distributions-oriented verification of probability forecasts for small data samples. Wea. Forecasting, 18, 903-917. http://dx.doi.org/10.1175/1520-0434(2003)018%3C0903:DVOPFF%3E2.0.CO;2
Bradley AA, Schwartz SS, Hashino T (2008) Sampling uncertainty and confidence intervals for the Brier score and Brier skill score. Wea. Forecasting, 23, 992-1006. doi: http://dx.doi.org/10.1175/2007WAF2007049.1
Brier GW (1950) Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 1-3. doi: http://dx.doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
Epstein ES (1969) A scoring system for probability forecasts of tanked categories. J. App. Met. Vol 8. No 6, 985-987. doi: http://dx.doi.org/10.1175/1520-0450(1969)008<0985:ASSFPF>2.0.CO;2
Ferro CAT (2007) Comparing probabilistic forecasting systems with the Brier score. Weather and Forecasting 22, 1076-1088. doi: http://dx.doi.org/10.1175/WAF1034.1
Ferro CAT, Richardson DS, Weigel AP (2008) On the effect of ensemble size on the discrete and continuous ranked probability scores. Meteorol. Appl., 15, 19-24. http://onlinelibrary.wiley.com/doi/10.1002/met.45/epdf
Ferro CAT, Stephenson DB (2011) Extremal Dependence Indices: improved verifiation measures for deterministic forecasts of rare binary events. Wea. Forecasting, 26, 699-713. doi: http://dx.doi.org/10.1175/WAF-D-10-05030.1
Ferro CAT, Fricker TE (2012) A bias-corrected decomposition of the Brier score. Quarterly Journal of the Royal Meteorological Society, 138, 1954-1960, doi:10.1002/qj.1924.
Ferro CAT (2014) Fair scores for ensemble forecasts. Quarterly Journal of the Royal Meteorological Society, 140, 1917-1923, http://dx.doi.org/10.1002/qj.2270
Gneiting T, Raftery AE (2007) Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association, 102, Issue 477, 359-378. doi: 10.1198/016214506000001437 http://www.eecs.harvard.edu/cs286r/courses/fall12/papers/Gneiting07.pdf
Hamill TM (2001) Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550-560. doi: http://dx.doi.org/10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2
Hersbach H (2000) Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather and Forecasting, 15, 559-570. doi: http://dx.doi.org/10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2
Hsu W-R., Murphy AH (1986) The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. Int. J. Forecasting, 2, 285-293. http://dx.doi.org/10.1016/0169-2070(86)90048-8
Jolliffe IT (2007) Uncertainty and inference for verification measures. Wea. Forecasting, 22, 637-650. doi: http://dx.doi.org/10.1175/WAF989.1
Jupp TE, Lowe R, Coelho CAS, Stephenson DB (2012): On the visualization, verification and recalibration of ternary probabilistic forecasts. Phil. Trans. R. Soc. A, 370, 1100-1120. doi:10.1098/rsta.2011.0350
Mason I (1982) A model for assessment of weather forecasts. Aust. Met. Mag., 30, 291-303. http://www.nssl.noaa.gov/users/brooks/public_html/feda/papers/mason82.pdf
Mason SJ, Weigel AP (2009) A generic forecast verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349. doi: http://dx.doi.org/10.1175/2008MWR2553.1
Mason SJ, Graham NE (2002) Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quarterly Journal of the Royal Meteorological Society Volume 128, Issue 584, pages 2145–2166, July 2002 Part B. DOI: 10.1256/003590002320603584 http://onlinelibrary.wiley.com/doi/10.1256/003590002320603584/pdf
Mason SJ (2008) Understanding forecast verification statistics. Meteorol. Appl., 15., 31-34. DOI: 10.1002/met.51 http://onlinelibrary.wiley.com/doi/10.1002/met.51/abstract
Murphy AH (1970) The ranked probability score and the probability score: A comparison. UDC 551.509.314. Vol. 98, No. 12. 917-924 http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.395.1780&rep=rep1&type=pdf
Murphy AH (1973) A new vector partition of the probability score. J. Appl. Meteor., 12, 595-600. doi: http://dx.doi.org/10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2
Murphy AH (1988) Skill scores based on the mean square error and their relationships to the correlation coefficient. Mon. Wea. Rev., 116, 2417-2424. doi: http://dx.doi.org/10.1175/1520-0493(1988)116<2417:SSBOTM>2.0.CO;2
Murphy AH (1993) What is a good forecast? An essay on the nature of goodness in weather forecasting. Wea. Forecasting, 8, 281-293. doi: http://dx.doi.org/10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2
Murphy AH (1995) The coefficients of correlation and determination as measures of performance in forecast verification. Wea. Forecasting, 10, 681-688. doi: http://dx.doi.org/10.1175/1520-0434(1995)010<0681:TCOCAD>2.0.CO;2
Murphy AH (1996) General decompositions of MSE-based skill scores: Measures of some basic aspects of forecast quality. Mon. Wea. Rev., 124, 2353-2369. doi: http://dx.doi.org/10.1175/1520-0493(1996)124<2353:GDOMBS>2.0.CO;2
Murphy AH, Epstein ES (1989) Skill scores and correlation coefficients in model verification. Mon. Wea. Rev., 117, 572-581. doi: http://dx.doi.org/10.1175/1520-0493(1989)117<0572:SSACCI>2.0.CO;2
Richardson DS (2000) Skill and relative economic value of the ECMWF ensemble prediction system. Quart. J. Royal Met. Soc., 126, 649-667. DOI: 10.1002/qj.49712656313 http://onlinelibrary.wiley.com/doi/10.1002/qj.49712656313/abstract
Roulston MS, Smith LA (2002) Evaluating probabilistic forecasts using information theory. Mon. Wea. Rev., 130, 1653-1660. DOI: http://dx.doi.org/10.1175/1520-0493(2002)130<1653:EPFUIT>2.0.CO;2
Stephenson DB, Casati C, Ferro CAT, Wilson CA (2008) The extreme dependency score: a non-vanishing measure for forecasts of rare events. Meteorol. Appl., 15, 41-50. DOI: 10.1002/met.53 http://onlinelibrary.wiley.com/doi/10.1002/met.53/abstract
Stephenson DB, Coelho CAS, Jolliffe IT (2008) Two extra components in the Brier Score Decomposition, Wea. Forecasting, 23, pp 752-757. doi: http://dx.doi.org/10.1175/2007WAF2006116.1
Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res., 106 (D7), 7183-7192. DOI: 10.1029/2000JD900719 http://onlinelibrary.wiley.com/doi/10.1029/2000JD900719/abstract
Weigel AP, Liniger MA, Appenzeller C (2007) The discrete Brier and ranked probability skill scores. Mon. Wea. Rev., 135, 118–124. doi: http://dx.doi.org/10.1175/MWR3280.1
Weigel AP, Liniger MA, Appenzeller C (2007) Generalization of the Discrete Brier and Ranked Probability Skill Scores for Weighted Multimodel Ensemble Forecasts. Mon. Wea. Rev., 135, 2778-2785. doi: http://dx.doi.org/10.1175/MWR3428.1
Weigel AP, Mason SJ (2011) The generalized discrimination Score for ensemble forecasts. Mon. Wea. Rev.,, 139, 3069-3074. doi: http://dx.doi.org/10.1175/MWR-D-10-05069.1
Wilson LJ, Burrows WR, Lanzinger A (1999) A strategy for verification of weather element forecasts from an ensemble prediction system. Mon. Wea. Rev., 127, 956-970. doi: http://dx.doi.org/10.1175/1520-0493(1999)127<0956:ASFVOW>2.0.CO;2
5) List of published literature on S2S verification
5.1) Assessment of S2S systems forecast skill
Hudson D, Alves O, Hendon HH, Marshall AG (2011) Bridging the gap between weather and seasonal forecasting: Intraseasonal forecasting for Australia. Quart. J. Roy. Meteor. Soc., 137,673–689, doi:10.1002/qj.769 http://onlinelibrary.wiley.com/doi/10.1002/qj.769/abstract
Hudson D, Marshall AG, Yin Y, Alves O, Hendon HH (2013) Improving intraseasonal prediction with a new ensemble generation strategy. Mon Wea Rev, 141, 4429-4449. doi: http://dx.doi.org/10.1175/MWR-D-13-00059.1
Hudson D, Marshall AG, Alves O, Shi L, Young G (2015) Forecasting upcoming extreme heat on multi-week to seasonal timescales: POAMA experimental forecast products. Bureau Research Report, No. 1. Bureau of Meteorology, Australia (http://www.bom.gov.au/research/research-reports.shtml).
Hudson D, Marshall AG, Alves O, Young G, Jones D, Watkins A (2015) Forewarned is forearmed: Extended range forecast guidance of recent extreme heat events in Australia. Weather and Forecasting. doi: http://dx.doi.org/10.1175/WAF-D-15-0079.1
Jung T, Miller MJ, Palmer TN (2010) Diagnosing the Origin of Extended-Range Forecast Errors. Mon. Wea. Rev., 138, 2434–2446. doi: http://dx.doi.org/10.1175/2010MWR3255.1
Koster RD, Mahanama SPP, Yamada TJ, Balsamo G, Berg AA, Boisserie M, Dirmeyer PA, Doblas-Reyes FJ, Drewitt G, Gordon CT, Guo Z, Jeong J.-H, Lawrence DM, Lee W.-S, Li Z, Luo L, Malyshev S, Merryfield WJ, Seneviratne SI, Stanelle T, van den Hurk BJJM, Vitart F, Wood EF (2010) Contribution of land surface initialization to subseasonal forecast skill: First results from a multi-model experiment. Geophysical Research Letters, Vol. 37, L02402, doi:10.1029/2009GL041677, 2010 http://onlinelibrary.wiley.com/doi/10.1029/2009GL041677/full
Kumar A, Chen M, Wang W (2011) An analysis of prediction skill of monthly mean climate variability. Clim. Dyn., 37, 1119-1131. http://link.springer.com/article/10.1007%2Fs00382-010-0901-4
Li S, Robertson AW (2015) Evaluation of Submonthly Precipitation Forecast Skill from Global Ensemble Prediction Systems. Monthly Weather Review 143:7, 2871-2889. doi:http://dx.doi.org/10.1175/MWR-D-14-00277.1
Marshall AG, Hudson D, Wheeler MC, Hendon HH, Alves O. (2012) Simulation and prediction of the Southern Annular Mode and its influence on Australian intra-seasonal climate in POAMA. Climate Dynamics. 38:2483-2502, doi:10.1007/s00382-011-1140-z. http://link.springer.com/article/10.1007%2Fs00382-011-1140-z
Marshall AG, Hudson D, Wheeler MC, Alves O, Hendon HH, Pook MJ, Risbey JS (2013) Intra-seasonal drivers of extreme heat over Australia in observations and POAMA-2. Climate Dynamics, doi: 10.1007/s00382-013-2016-1 http://link.springer.com/article/10.1007%2Fs00382-013-2016-1
Marshall AG, Hudson D, Hendon HH, Pook MJ, Alves O, Wheeler MC (2014) Simulation and prediction of blocking in the Australian region and its influence on intra-seasonal rainfall in POAMA-2. Clim. Dyn., 42, 3271-3288, DOI: 10.1007/s00382-013-1974-7 http://link.springer.com/article/10.1007%2Fs00382-013-1974-7
Mastrangelo D, Malguzzi P, Rendina C, Drofa O, Buzzi A (2012) First Outcomes from the CNR-ISAC monthly forecasting system Adv. Sci. Res., 8, 77–82, 2012 www.adv-sci-res.net/8/77/2012/ doi:10.5194/asr-8-77-2012
Vitart F (2014): Evolution of ECMWF sub-seasonal forecast skill scores. Quarterly Journal of the Royal Meteorological Society. Volume 140, Issue 683, pages 1889–1899, July 2014 Part B. DOI: 10.1002/qj.2256 http://onlinelibrary.wiley.com/doi/10.1002/qj.2256/abstract
Vitart F (2013) Evolution of ECMWF sub-seasonal forecast skill scores over the past 10 years . ECMWF Technical report http://www.ecmwf.int/sites/default/files/elibrary/2013/12932-evolution-ecmwf-sub-seasonal-forecast-skill-scores-over-past-10-years.pdf
Weigel A, Baggenstos D, Liniger MA, Vitart F, Appenzeller C. (2008) Probabilistic verification of monthly temperature forecasts. Mon. Weather Review 136: 5162–5182. doi: http://dx.doi.org/10.1175/2008MWR2551.1
White CJ, Hudson D, Alves O (2013) ENSO, the IOD and the intraseasonal prediction of heat extremes across Australia using POAMA-2. Climate Dynamics. doi:10.1007/s00382-013-2007-2 http://link.springer.com/article/10.1007%2Fs00382-013-2007-2
5.2) Assessment of MJO/ISO forecast skill
Gottschalck J, Wheeler M, Weickmann K, Vitart F, Savage N, Lin H, Hendon H, Waliser D, Sperber K, Nakagawa M, Prestrelo C, Flatau M, Higgins W (2010) A Framework for Assessing Operational Madden–Julian Oscillation Forecasts: A CLIVAR MJO Working Group Project. Bull. Amer. Meteor. Soc., 91, 1247–1258. doi: http://dx.doi.org/10.1175/2010BAMS2816.1
Jiang X, Waliser DE, Wheeler MC, Jones C, Lee M.-I, Schubert SD (2008) Assessing the skill of an all-season statistical forecast model for the Madden-Julian oscillation. Mon. Wea. Rev., 136, 1940-1956.
doi: http://dx.doi.org/10.1175/2007MWR2305.1
Jones C, Waliser DE, Schemm JKE, Lau WKM (2000) Prediction skill of the Madden and Julian oscillation in dynamical extended-range forecasts. Clim. Dyn. 16: 273–289. http://link.springer.com/article/10.1007/s003820050327
Kim H, Webster PJ, Toma VE, Kim D (2014) Predictability and prediction skill of the MJO in two operational forecasting systems, J. Climate, 27 (14), 5364‐5378. doi: http://dx.doi.org/10.1175/JCLI-D-13-00480.1
Lee S-S, Wang B, Waliser DE, Neena JM, Lee J-Y (2015) Predictability and prediction skill of the boreal summer intraseasonal oscillation in the Intraseasonal Variability Hindcast Experiment. Climate Dynamics 45, 2123-2135. http://link.springer.com/article/10.1007%2Fs00382-014-2461-5
Liess S, Waliser DE, Schubert S. 2005. Predictability studies of the intraseasonal oscillation with the ECHAM5 GCM. J. Atmos. Sci. 62: 3320–3336. doi: http://dx.doi.org/10.1175/JAS3542.1
Lin H, Brunet G, Derome J (2008) Forecast Skill of the Madden–Julian Oscillation in Two Canadian Atmospheric Models. Mon. Wea. Rev., 136, 4130–4149. doi: http://dx.doi.org/10.1175/2008MWR2459.1
Lin H, Brunet G, Fontecilla JS (2010) Impact of the Madden-Julian Oscillation on the intraseasonal forecast skill of the North Atlantic Oscillation. Geophys. Res. Lett.,37,L19803, doi:10.1029/2010GL044315 http://onlinelibrary.wiley.com/doi/10.1029/2010GL046131/pdf
MacLachlan, C., Arribas, A., Peterson, K. A., Maidens, A., Fereday, D., Scaife, A. A., Gordon, M., Vellinga, M., Williams, A., Comer, R. E., Camp, J., Xavier, P. and Madec, G. (2015) Global Seasonal forecast system version 5 (GloSea5): a high-resolution seasonal forecast system. Q.J.R. Meteorol. Soc., 141: 1072–1084. doi: 10.1002/qj.2396 http://onlinelibrary.wiley.com/doi/10.1002/qj.2396/abstract
Maharaj EA, Wheeler MC (2005) Forecasting an index of the Madden-Oscillation. Int. J. Climatol. 25: 1611–1618 (2005) DOI: 10.1002/joc.1206 http://onlinelibrary.wiley.com/doi/10.1002/joc.1206/full
Marshall AG, Hudson D, Wheeler MC, Hendon HH, Alves O (2010) Assessing the simulation and prediction of rainfall associated with the MJO in the POAMA seasonal forecast system. Climate Dynamics, 37, 2129-2141, doi: 10.1007/s00382-010-0948-2 http://link.springer.com/article/10.1007%2Fs00382-010-0948-2
Marshall AG, Hudson D, Wheeler MC, Alves O, Hendon HH, Pook MJ, Risbey JS (2014) Intra-seasonal drivers of extreme heat over Australia in observations and POAMA-2. Climate Dynamics, 43, 1915-1937, DOI: 10.1007/s00382-013-2016-1 http://link.springer.com/article/10.1007%2Fs00382-013-2016-1
Neena JM, Lee JY, Waliser D, Wang B, Jiang X (2014): Predictability of the Madden–Julian Oscillation in the Intraseasonal Variability Hindcast Experiment (ISVHE). J. Climate, 27, 4531–4543. doi: http://dx.doi.org/10.1175/JCLI-D-13-00624.1
Rashid HA, Hendon HH, Wheeler MC, Alves O (2011) Prediction of the Madden–Julian oscillation with the POAMA dynamical prediction system. Climate Dynamics. Volume 36, Issue 3-4, pp 649-661 http://link.springer.com/article/10.1007%2Fs00382-010-0754-x
Vitart F, Molteni F (2010) Simulation of the Madden- Julian Oscillation and its teleconnections in the ECMWF forecast system. Quarterly Journal of the Royal Meteorological Society 136:10.1002/qj.v136:649, 842-855. http://onlinelibrary.wiley.com/doi/10.1002/qj.623/pdf
5.3) Assessment of monsoon systems forecast skill and associated characteristics
Drosdowsky W, Wheeler MC (2014): Predicting the onset of the north Australian wet season with the POAMA dynamical prediction system. Wea. Forecasting, 29, 150-161. doi: http://dx.doi.org/10.1175/WAF-D-13-00091.1
Jones C, Carvalho LMV, Liebmann B (2012) Forecast Skill of the South American Monsoon System. J. Climate, 25, 1883–1889. doi: http://dx.doi.org/10.1175/JCLI-D-11-00586.1
Jones C, Carvalho LMV (2002) Active and Break Phases in the South American Monsoon System. J. Climate,15, 905–914 doi: http://dx.doi.org/10.1175/1520-0442(2002)015<0905:AABPIT>2.0.CO;2
Lo F, Wheeler MC, Meinke H, Donald A (2007) Probabilistic Forecasts of the Onset of the North Australian Wet Season. Mon. Wea. Rev., 135, 3506–3520. doi: http://dx.doi.org/10.1175/MWR3473.1
Moron V, Robertson AW, Boer R (2009) Spatial Coherence and Seasonal Predictability of Monsoon Onset over Indonesia. J. Climate, 22, 840–850. doi: http://dx.doi.org/10.1175/2008JCLI2435.1
Muñoz ÁG, Goddard L, Mason SJ, Robertson AW (2016) Cross-timescale interactions and rainfall extreme events in South East South America for the austral summer. Part II: Predictive skill. J. Climate. doi: http://dx.doi.org/10.1175/JCLI-D-15-0699.1
Vellinga M, Arribas A, Graham R (2013) Seasonal forecasts for regional onset of the West African monsoon. Climate Dynamics. Volume 40, Issue 11, pp 3047-3070. http://link.springer.com/article/10.1007%2Fs00382-012-1520-z
5.4) Applications
Calanca P, Bolius D, Weigel AP, Liniger MA (2011) Application of long-range weather forecasts to agricultural decision problems in Europe. The Journal of Agricultural Science 149, 15-22. http://dx.doi.org/10.1017/S0021859610000729
Hirschi M, Spirig C, Weigel AP, Calanca P, Samietz J, Rotach MW (2012) Monthly Weather Forecasts in a Pest Forecasting Context: Downscaling, Recalibration, and Skill Improvement. J. Appl. Meteor. Climatol., 51, 1633–1638. doi: http://dx.doi.org/10.1175/JAMC-D-12-082.1
Lynch KJ, Brayshaw DJ, Charlton-Perez A (2014) Verification of European Subseasonal Wind Speed Forecasts. Mon. Wea. Rev., 142, 2978–2990.
doi: http://dx.doi.org/10.1175/MWR-D-13-00341.1
Spillman CM, Hartog JR, Hobday AJ, Hudson DA (2015) Predicting environmental drivers for prawn aquaculture production to aid improved farm management. Aquaculture. 447: 56-65. doi:10.1016/j.aquaculture.2015.02.008
5.5) Seamless verification
Zhu H, Wheeler MC, Sobel AH, Hudson D (2014) Seamless Precipitation Prediction Skill in the Tropics and Extratropics from a Global Model. Mon. Wea. Rev., 142, 1556–1569. doi: http://dx.doi.org/10.1175/MWR-D-13-00222.1
6) Available reference verification datasets for assessing S2S forecast quality
6.1) Atmospheric parameters (e.g. geopotential height, temperature, SLP, wind, etc)
| Reanalysis | Source | References |
| 20CR Access via IRI Data Library | NOAA CIRES | Compo et al. (2006) |
| ERA-Interim | ECMWF | Dee et al. (2011) |
| ERA-20C | ECMWF | Poli et al. (2015) |
| ERA-20CM | ECMWF | Hersbach et al. (2015) |
| JRA-55 | JMA | Kobayashi et al. (2015) |
| MERRA Access via IRI Data Library | NASA | Rienecker et al. (2011) |
| MERRA-2 Access via IRI Data Library | NASA | Bosilovich et al. (2015) |
| NCEP NCAR | Kalnay et al. (1996) | |
| NCEP NCAR | Kanamitsu et al. (2002) | |
| NCEP CFSR | NCEP NCAR | Saha et al. (2010) |
Additional information about all reanalysis listed above is available at theWeb-based Reanalysis Intercomparison Tools (WRIT)
6.2) Oceanic parameters
| Variable | Name | Source | Characteristics | References |
| Sea Surface temperature | NOAA NCDC | Monthly (2 x 2 degrees in lat and lon) | ||
| Sea Surface temperature | NOAA NCDC | Monthly (2 x 2 degrees in lat and lon) | ||
| Sea Surface temperature | NOAA NCEP | Weekly/Monthly (1 x 1 degree in lat and lon) | Reynolds et al. (2002) | |
| Sea Surface temperature | NOAA NCDC | Daily 0.25 x 0.25 degree in lat and lon) | ||
| Sea Surface temperature | ECMWF | Daily 0.7 x 0.7 degree in lat and lon) | Dee et al. (2011) | |
| Sub-surface ocean parameters | NCEP | Monthly 0.333 x 1.0 degree in lat and lon) | See this link |
6.3) Surface parameters
| Variable | Name | Source | Characteristics | References |
| Surface air temperature | NOAA NCEP CPC | Monthly (0.5 x 0.5 degrees in lat and lon) | Fan and van Den Dool (2008) | |
| Surface air temperature | Univ. Delaware | Monthly (0.5 x 0.5 degrees in lat and lon) | See this link | |
| Surface air temperature | NOAA-NCDC/WMO | Daily : 33147 stations (max, mean, min)
| Vose et al. (1992) | |
| Precipitation | NOAA NCEP CPC | Daily 0.5 x 0.5 degree in lat and lon) | ||
| Precipitation | NOAA NCEP CPC | 3hr/Daily 0.25 x 0.25 degree in lat and lon) | Joyce et al. (2004) | |
| Precipitation | UCSB | Daily/dekad/monthly 0.25 x 0.25 degree in lat and lon) | Funk et al. (2014) | |
| Precipitation | Univ. Delaware | Monthly (0.5 x 0.5 degrees in lat and lon) | See this link | |
| Precipitation | NOAA-NCDC/WMO | Daily : 33147 stations (max, mean, min)
| Vose et al. (1992) | |
| Precipitation | NOAA NCEP CPC | Pentad/Monthly (2.5 x 2.5 degrees in lat and lon) | Xie and Arkin (1997) | |
| Precipitation | DWD | Daily/Monthly (1.0 x 1.0 degrees in lat and lon) | ||
| Precipitation | NOAA NCEP CPC | Monthly (2.5 x 2.5 degrees in lat and lon) | Xie and Arkin (1997) |
6.4) Datasets accessible via the KNMI Climate Explorer
| Daily station data | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Monthly station data | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Daily gridded fields | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Monthly gridded observations | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Monthly reanalysis fields | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
7) S2S project models A total of 11 models are currently contributing to the S2S project data archive hosted at ECMWF (see documentation). The main features of the forecasts and re-forecasts of these 11 models are included in the table below:
7.1) Accessing S2S models data S2S forecast and re-forecast data are available through ECMWF and CMA.
7.2) Visualizing S2S forecast products S2S forecast products from 6 S2S models (BoM, CMA, NCEP, UKMO, ECMWF, JMA) generated every Thursday since 7 January 2016 are available through the ECMWF forecast products page.
Additional forecast products are available at the S2S Museum operated for the promotion of utilization of S2S data by Dr. Mio Matsueda (University of Tsukuba and University of Oxford). The currently available products include ensemble forecasts for specific atmospheric phenomena such as: - Artic/Antartic Oscillation index
|