 |
Click here
to view the AFC Powerpoint Presentation
2005-2006 ISU AMS Forecast Contest Rules
Forecast Calendar
The contest will consist of 22 2-week and 1-week periods over a year, with the final period as the year-end tournament. Each 2-week period will consist of 4 days per week (Monday through Thursday), for a total of 8 forecasts per period. Forecasters must complete at least 12 of the 21 regular season periods to have their cumulative score count in the final standings and to be eligible to participate the year-end tournament. In each 2-week period, forecasters must submit a forecast on at least 6 of the 8 days in the period, to have their forecasts counted for that period. In a 1-week period, forecasters must submit a forecast on at least 3 of the 4 days in the period. When a forecaster misses 10 of the 21 periods, they will be disqualified from the contest.Forecast Interval
The forecast day will consist of a different time period that will change from period to period. However, the forecast interval for each forecast element will stay the same, regardless of the day of week the forecast is for. For precipitation forecasts, the nighttime period runs from 0000 to 1200 UTC. The daytime period runs from 1200 to 0000 UTC. For temperature forecasts, the nighttime period will consist of the minimum temperature between 7 pm and 8 am LST, or 0100 to 1400 UTC. The daytime period will consist of the maximum temperature between 7 am and 7 pm LST, or 1300 to 0100 UTC. For aviation forecasts, the forecast interval will run from 0300 to 1800 UTC.Elements of the Forecast
The forecast will consist of two distinct types of forecast elements. The majority of forecasts will be made for the basic public elements of temperature and precipitation. A minority of forecasts will be made for the basic aviation elements of wind, ceiling height, and visibility.
For the public forecast, depending on the time period the forecast is for, the following elements may be forecast: the nighttime minimum and daytime maximum temperatures, the categorized probability of precipitation (POP), the categorized amount of quantitative precipitation forecast (QPF), and the categorized amount of quantitative snowfall forecast (QSF).
Temperatures will be forecast to the nearest whole degree Fahrenheit. Errors will be capped at 15 degrees.
POP categories are:
0 -- 0 to 4% 1 -- 5 to 14% 2 -- 15 to 24% 3 -- 25 to 34% 4 -- 35 to 44% 5 -- 45 to 54% 6 -- 55 to 64% 7 -- 65 to 74% 8 -- 75 to 84% 9 -- 85 to 94% 10 -- 95 to 100%
QPF categories are:
0 -- 0.00" or trace 1 -- 0.01" to 0.09" 2 -- 0.10" to 0.24" 3 -- 0.25" to 0.49" 4 -- 0.50" to 0.99" 5 -- 1.00" to 1.99" 6 -- >= 2.00"
QSF categories are:
0 -- 0.0" or trace 1 -- 0.1" to 1.4" 2 -- 1.5" to 3.4" 4 -- 3.5" to 5.4" 6 -- 5.5" to 7.4" 8 -- >= 7.5"
For the aviation forecast, the following elements will be forecast: the categorized highest sustained wind speed (HWS), the categorized lowest reported prevailing ceiling (LRC), and the categorized lowest reported prevailing visibility (LRV).
HWS categories are:
1 -- 0 to 3 kts 2 -- 4 to 7 kts 3 -- 8 to 11 kts 4 -- 12 to 15 kts 5 -- 16 to 19 kts 6 -- 20 to 23 kts 7 -- 24 to 27 kts 8 -- >= 28 kts
LRC categories are:
1 -- < 200 feet 2 -- 200 to 400 feet 3 -- 500 to 900 feet 4 -- 1000 to 1900 feet 5 -- 2000 to 3000 feet 6 -- 3100 to 6500 feet 7 -- 6600 to 12000 feet 8 -- >= 12000 feet or no ceiling
LRV categories are:
1 -- < 0.5 miles 2 -- 0.5 to 0.9 miles 3 -- 1.0 to 1.9 miles 4 -- 2.0 to 2.9 miles 5 -- 3.0 to 5.4 miles 6 -- 5.5 to 6.4 miles 7 -- >= 6.5 miles Verification
All elements (except for snowfall) will be verified using the official METARs produced by ASOSs at the site the forecast is for. For snowfall, the official National Weather Service cooperative observer for that city will be used.
Temperatures will be verified based on the 6 hour maximum and minimum temperature reported at standard synoptic observation times. Since the forecast interval does not exactly match the 6 hour blocks, the Chairperson shall make a reasonable judgment based on hourly observations and the corresponding 6 hour blocks of time.
POP will be verified as either 0 or 100%. 100%, if at least 0.01" of precipitation falls during the forecast interval. For both automated and manual sites, the 6 hour precipitation and snowfall accumulation at standard synoptic observation times will be used for QPF and QSF. Both POP and QPF will be verified using the automated report during rainfall events. When snow and/or sleet is reported at anytime by the automated weather identifier during the time period, the official NWS cooperative observer site for that city will be used. If unknown precipitation is reported, the Chairperson shall make a reasonable judgment on whether this precipitation was snow and/or sleet. If a daily report must be used, the ASOS observations will be used to gauge how much precipitation fell in a given time period.
The HWS will be verified from the highest sustained wind speed reported in the time period. This does not apply to wind gusts or the peak wind gust. Both hourly and special observations will be used for verification.
The LRC and LRV will be verified based on any observations (hourly or special) reported in the forecast interval. However, if a variable ceiling or visibility is reported in remarks, and this value is lower than any non-variable ceiling or visibility, the variable range will not be used. Only the prevailing lowest reported ceiling or visibility will be used.
In the event of failure with any of the automated equipment, maintenance performed on the site sensors, any missing observations, or any value deemed unrepresentative of what actually occurred, the Chairperson shall perform a reasonable decision regarding the parameter in question based on the nearest weather data available. Questionable parameters will be discussed via email with forecast participants, along with the scientific reasoning that necessitated the change. Any objection to a change should be voiced or written via email to the Chairperson. Based on differing evidence or sound scientific reasoning, the Chairperson may amend the verification.Scoring of Daily Forecasts
All forecast elements will use the square of the error. This will cause a greater differentiation between the most accurate forecasts versus the least accurate ones. All squared errors will be divided by a normalization factor, in an effort to equally weight the different types of categories with each other.
Temperature
0 degF = 0 errors divide by 5 = 0.0 1 degF = 1 errors divide by 5 = 0.2 2 degF = 4 errors divide by 5 = 0.8 3 degF = 9 errors divide by 5 = 1.8 4 degF = 16 errors divide by 5 = 3.2 5 degF = 25 errors divide by 5 = 5.0 6 degF = 36 errors divide by 5 = 7.2 7 degF = 49 errors divide by 5 = 9.8 8 degF = 64 errors divide by 5 = 12.8 9 degF = 81 errors divide by 5 = 16.2 10 degF = 100 errors divide by 5 = 20.0 11 degF = 121 errors divide by 5 = 24.2 12 degF = 144 errors divide by 5 = 28.8 13 degF = 169 errors divide by 5 = 33.8 14 degF = 196 errors divide by 5 = 39.2 15 degF + = 225 errors divide by 5 = 45.0
POP
0% = 0 errors divide by 2.5 = 0.0 10% = 1 errors divide by 2.5 = 0.4 20% = 4 errors divide by 2.5 = 1.6 30% = 9 errors divide by 2.5 = 3.6 40% = 16 errors divide by 2.5 = 6.4 50% = 25 errors divide by 2.5 = 10.0 60% = 36 errors divide by 2.5 = 14.4 70% = 49 errors divide by 2.5 = 19.6 80% = 64 errors divide by 2.5 = 25.6 90% = 81 errors divide by 2.5 = 32.4 100% = 100 errors divide by 2.5 = 40.0
QPF
0 cat = 0 errors divide by 0.5 = 0.0 1 cat = 1 errors divide by 0.5 = 2.0 2 cat = 4 errors divide by 0.5 = 8.0 3 cat = 9 errors divide by 0.5 = 18.0 4 cat = 16 errors divide by 0.5 = 32.0 5 cat = 25 errors divide by 0.5 = 50.0 6 cat = 36 errors divide by 0.5 = 72.0
QSF
0 cat = 0 errors divide by 0.5 = 0.0 1 cat = 1 errors divide by 0.5 = 2.0 2 cat = 4 errors divide by 0.5 = 8.0 3 cat = 9 errors divide by 0.5 = 18.0 4 cat = 16 errors divide by 0.5 = 32.0 5 cat = 25 errors divide by 0.5 = 50.0 6 cat = 36 errors divide by 0.5 = 72.0 7 cat = 49 errors divide by 0.5 = 98.0 8 cat = 64 errors divide by 0.5 = 128.0
HWS
0 cat = 0 errors divide by 1 = 0.0 1 cat = 1 errors divide by 1 = 1.0 2 cat = 4 errors divide by 1 = 4.0 3 cat = 9 errors divide by 1 = 9.0 4 cat = 16 errors divide by 1 = 16.0 5 cat = 25 errors divide by 1 = 25.0 6 cat = 36 errors divide by 1 = 36.0
LRC
0 cat = 0 errors divide by 1 = 0.0 1 cat = 1 errors divide by 1 = 1.0 2 cat = 4 errors divide by 1 = 4.0 3 cat = 9 errors divide by 1 = 9.0 4 cat = 16 errors divide by 1 = 16.0 5 cat = 25 errors divide by 1 = 25.0 6 cat = 36 errors divide by 1 = 36.0 7 cat = 49 errors divide by 1 = 49.0
LRV
0 cat = 0 errors divide by 1 = 0.0 1 cat = 1 errors divide by 1 = 1.0 2 cat = 4 errors divide by 1 = 4.0 3 cat = 9 errors divide by 1 = 9.0 4 cat = 16 errors divide by 1 = 16.0 5 cat = 25 errors divide by 1 = 25.0 6 cat = 36 errors divide by 1 = 36.0 Submission of Forecasts
Forecasts must be submitted to the Chairperson by 2330 UTC using Portfolio, or email if Portfolio is inaccessible. There is a 5 minute grace period for forecasts submitted by email after the deadline. No forecast will be accepted if dated after 2335 UTC.Failure to Forecast
Forecasters who do not make a forecast will be assigned consensus MOS values.Occasional Planned Absences
If a forecaster knows in advance that he/she cannot forecast for the next day (or days), two options are available:- he/she may do nothing, and take consensus MOS as a forecast
- he/she may make a planned guidance-based forecast (for periods beyond 24h)
Computer Model Guidance Forecasters
GFS MOS (MAV/MEX/MEX Ensemble Consensus[MEC]), Eta MOS (MET), NGM MOS (FWC), Consensus MOS (CMO), and Consensus MAV/MET (MAVMET) forecast values from their latest runs will be inputted by the Chairperson or designee. Since the MOS output may not directly correspond with the time interval and elements of the forecast contest for all periods, the MOS will be required to opt-out of certain periods.
For the MAV and MET, they will forecast in all periods through day 2.5. The FWC will only be used through day 2. The MEX and MEC will not forecast in the two aviation forecast periods.
CMO will be the average of all MOS forecasts rounded to the same precision. MAVMET will be the average of the MAV and MET forecasts rounded to the same precision nearest the MET. In the case of parameters that are not forecast for any MOS element, the consensus of the other MOS will be used for input.NWS Forecaster
The NWS official forecasts issued through 2340 UTC will be inputted by the Chairperson or designee. The Point Forecast Matrix (PFM) will be used for the temperature, POP, QPF, and QSF elements. The Terminal Aerodrome Forecast (TAF) will be used for LRC and LRV elements. The highest wind speed value in either the PFM or TAF will be used for the HWS element.Unreasonable Forecasts
Prank forecasts will not be tolerated. To avoid having obviously non-serious forecasts contaminate consensus values, forecasts judged unreasonable will be eliminated from the consensus value computation. Furthermore, forecasts judged to be unreasonable will be converted to guidance forecasts, and the forecaster marked absent for the day. The Chairperson shall perform the unreasonable forecast rejection.Dropping Out of the Contest
Participants who decide to withdraw from the contest must do so by sending an email statement to that effect to the Chairperson. The withdrawal will take effect at the end of the forecast period. Forecasters who withdraw in the midst of a forecast period receive guidance forecasts until the end of the forecast period. Rule 2 shall be applied to determine whether or not that person's score counts.Normalized Scores
Since schedules vary, forecasters will not all choose to participate in the same periods. This means that the sample of forecast days will not be the same for all forecasters. An attempt will be made to adjust for any differences in difficulty between different periods and obtain normalized scores. A normalization procedure will be used which is based upon standardized departures from the mean:
Normalized score = [(raw score - mean)*10]/S + 80, S is the standard deviation of all scores for the period, "mean" is the mean of all non-absent forecasts for the period, the factor 10 is a normalized standard deviation, and 80 is a standardized mean consensus score.
Example: Suppose that the mean for the period is 81, the standard deviation is 15, and a forecaster's score is 67 (which is almost one standard deviation better than the mean). The normalized score is 70.7. The cumulative normalized score at the end of the contest, then, will represent the number of standard deviations that the forecaster averaged above or below consensus.Year-End Tournament
The last city of the contest year will be designated the year-end tournament. The tournament will consist of head-to-head weekly battles between two forecasters. The winner will be determined by the better normalized score during the week. Seeding is based on the final regular season cumulative normalized score. The winners of each group will move on and play the other winners, while the losers play the other losers.