Keep Me Honest

2013-2014 Colorado

Welcome to the 2013-2014 season! If you need an introduction to this page, scroll down as I talk about my methods in the sections below.

Forecast 2014-04-12-KMH

Forecast 2014-03-30-KMH.xlsx copy

Forecast 2014-03-26-KMH

Forecast 2014-01-26-KMH

Forecast 2014-01-03-KMH

Forecast 2013-12-31-KMH

2013-12-07

2013-12-03

2012-2013 Colorado

I love keeping track of my accuracy because it helps me to be a better forecaster and publishing the data shows you that we’re fully transparent and try our hardest to make the right forecast. What I show here is my forecast for Colorado compared to what actually happened. I also save forecasts from other weather companies, but it’s hard to compare the accuracy between different companies because some only forecast snow for 48 hours, some 72 hours, and I forecast for 5 days (120 hours). We hope to one day create an automated system to show accuracy statistics, but until then it’s all manual. Also, this is only available for Colorado since it takes more time than I expect our other forecasters to put into it. Rest assured though that they track their accuracy internally and are always trying to get better…we never like to be wrong!  – JOEL GRATZ, 11/12/2012

*** Please note that the forecast range is the addition for many ranges, ie. “Tonight: 2-4 inches, Tomorrow, 4-6 inches” is a total range of 6-10″. It may seem like the final ranges shown below are wide, and this is because summing many small ranges creates a wider range. ***

The image below compares various forecasts. Click to enlarge. Please note that this is a special case with a short-duration storm and often it’s not possible to compare forecasts because some forecasters only make forecasts for short lead times. For example, if a storm is going to last for two days starting “tonight”, this means the snow from the storm could fall for up to 60 hours in advance (a two-day storm = 48hrs and it’s another 12hrs between making the forecast this morning and the start of the storm, so 48+12=60 hours). The National Weather Service only forecasts snow out to 48 hours, CAIC out to 36 hours, and The Weather Channel out to 48 hours. In that situation, we couldn’t compare those forecasts with the longer-duration forecasts (out to 5 days for OpenSnow, 6 days for SnowForecast.com, and 6 days for AspenWeather.net). For this storm, it only lasted for about 24 hours, so all forecasts could be compared.


2011-2012 Colorado

I kept accuracy statistics during the 2011-2012 season for my Colorado forecasts though have not posted them yet because they require a lot of formatting to make them readable and understandable. Sorry for the delay, and I’m not sure if I will actually get to this or not as things are getting busy. Rest assured that they are similar to the previous season (2010-2011). – JOEL GRATZ, 11/12/2012


Long-time readers will remember my “Correct”, “Pretty Good”, and “Wrong” accuracy tracking for the past two seasons. I am doing it differently in 2010-2011 since I am making a forecast every day. Over the past two years, I made forecasts every few days, and “counted” the 7-day forecasts made on Thursday as the official forecasts for the week. Since I am making forecasts every day this season, I am instead going to track the accuracy of the exact snow forecast made the day before the storm. I am only tracking storms that are defined events that start and stop over one or a few days.


2010-2011 Colorado: Total Accuracy

I forecast a range of snow for each resort for each storm (example: 5-10 inches). If the storm total snowfall for that resort is in the range, I count it as “Correct”. If the storm total snowfall is outside of the range, I count it as “Incorrect”. I then calculate the error for each forecast for each resort (example: forecast was for 5-10 inches and 12 inches actually fell – this is an error of 2 inches). I total the errors for all resorts and then rank each forecasting service. The service with the lowest total error comes in 1st place.

* I do not count all storms for “1st place finishes”. Some storms either span a long time period that makes comparing forecasts impossible, or for a few storms I was not able to copy other forecasts before they were updated. I only count a “1st place finish” if the forecasts were comparable (produced at the same time and for the same duration in the future).


Here are the individual storms for the 2010-2011 season in reverse chronological order:

STORM #23: April 2-4, 2011


STORM #22: March 27-29, 2011


STORM #21: March 17-18, 2011


STORM #20: March 7-March 8, 2011


STORM #19: March 3-March 4, 2001


STORM #18: February 25-February 26, 2011


STORM #17: February 19-February 20, 2011


STORM #16: February 16-February 17, 2011


STORM #15: February 05-February 08, 2011

Almost every resort measured more snow than was forecast – I love it when that happens! Because the storm was five days in length, I could only compare my forecast to two other forecasting services since the NWS, CAIC, and TWC only forecast snow for 36-48 hours.


STORM #14: January 30-February 1, 2011


STORM #13: January 21-23, 2011


STORM #12: January 19, 2011


STORM #11: January 16-18, 2011


STORM #10: January 8-10, 2011

An equal-opportunity storm for most of Colorado. While Steamboat and the northern / western mountains had the most snow, there was something for everyone. The San Jauns were far from shut out, and some easterly upslope threw down some freshies for Eldora and Echo as well. My forecast was pretty good once again, and came out in 1st place among five other snow forecasting services. You’ll notice that my average range of snowfall is large than others, and I’m happy with this. I DO NOT forecast a large range to make my numbers look more accurate. The range I provide is the best forecast I can make, and I don’t think a range of only 2-3 inches is scientifically accurate when snow totals could be over a foot. There is just too much variability and randomness in snow forecasting to have high confidence in a tiny possible range of snowfall.


STORM #9: December 29-31, 2010

This storm was all about the San Juan mountains, and was largely a dud in the central and northern mountains with snow totals coming in under the forecast or at the lower end of the range. Some of the storm’s energy jumped from western Colorado to the eastern plains and bypassed many of the mountains. Oh well – can’t win ‘em all!

Also, I wasn’t able to save the forecasts from other forecasters at the beginning of the storm, so I have no chart to compare my forecast versus others. Sorry! My guess is that I overforecast the storm like others did, but possibly by a bit more.


STORM #8: December 18-23, 2010

The “(kinda) Biggest storm EVER” kept flakes in the air for almost a week, with the biggest totals in the central and southern mountains. Accuracy was pretty good for this storm, though some of the I-70 and north areas ended up on the lower end of the range.


STORM #7: December 15-16, 2010

A nice mid-week powder day was had by most!


STORM #6: December 10-11, 2010

What fantastic powder from I-70 down to Aspen. On the second chart, I added the average range of snow forecast by each entity.


STORM #5: December 6-7, 2010

This storm over performed for areas along I-70 like Vail, Copper, Breckenridge, and Loveland. Good job to them! Otherwise, I’m back in the lead for this storm, and now have first-place finishes in four of the five storms this season.


STORM #4: November 28-30, 2010

While my forecasts were tops in the state for the last three storms, I slipped to third place for this most recent storm. The forecasters at the National Weather Service and Colorado Avalanche Information Center should be commended for some great work, however there are a few differences between their forecasts and what I provide. The NWS only forecasts snow amounts out to 48 hours (and CAIC only out 36 hours), which means you need to wait until the day before the storm begins to see snowfall forecasts for an entire storm, which usually lasts at least 24-48 hours. My snowfall forecasts go out to 120 hours (5 days).

The forecasts I compare below were made just a few hours before the storm, but I had essentially the same forecast posted a full 2 days prior on Friday morning (11/26) which resulted in only a slightly higher average error per resort of 1.7 inches. Basically, you could “see the storm coming” a full two days earlier with my forecasts.


STORM #3: November 21-24, 2010


STORM #2: November 8-10, 2010


STORM #1: October 24-27, 2010


Accuracy for 2009-2010

Correct = right on with the forecast.

Pretty Good = the intent of the forecast was correct (snow, sun, wind, etc), but maybe things were a little different than expected (more or less snow, sun, wind, etc).

Wrong = dammit, the forecast was terrible.  Watcha gonna do?

 

Click here to download a spreadsheet (.xls) of my 2009-2010 accuracy.


Seasonal Accuracy for 2008-2009

Do you believe this statement?  “I am the most accurate weather forecaster in Colorado.”

Yep – it’s a bold statement.  But I don’t know of anyone else who keeps statistics on the accuracy of their forecasts.  Since nobody can prove me wrong…I’ll consider myself the most accurate weather guy around.  Maybe this need for attention is part of my ‘OCS’ (only child syndrome)…  I have descriptions of each day’s forecast and outcome – shoot me an email if you’d like to see the details!

Here are the stats for my 168 days of forecasting during the 2008-2009 ski season in Colorado.  There is nothing fancy here.  I made the forecasts each Thursday night, and predicted weather for the next 7 days.  After those 7 days, I looked at the actual weather and ranked my forecasts:

Correct = right on with the forecast.

Pretty Good = the intent of the forecast was correct (snow, sun, wind, etc), but maybe things were a little different than expected (more or less snow, sun, wind, etc).

Wrong = dammit, the forecast was terrible.  Watcha gonna do?

Admittedly, this is not a very scientific way to assess the accuracy of the forecast.  But it’s a good start – and I invite others to start keeping track of their accuracy!

Hmmm…maybe there’s something to the old adage that “…weathermen are only 50% right…”.