I was a guest on the Jay Thomas Show on WDAY radio out of Fargo today. The guest host was Rob Port from the SayAnythingBlog, Forum Communications Op-Ed pages, TV appearances, and probably a bunch of stuff I am forgetting. He might be getting close to the title of “King of All North Dakota Media” at this point. The topic was forecasting. Yes I know. Friday afternoon in the summer and we were talking forecasting. I do not apologize for it, since I am pretty much always thinking about statistical models.

Rob and I are in agreement that forecast accuracy is vital if a forecast is to help the legislature do its job. We lamented the errors in forecasts that seemed ignored over the past 5-7 years, but it only recently hit me what I find so aggravating about the whole situation.

In any worthwhile forecasting exercise it is really unlikely that you are going to get the number spot on. Ever. You want to minimize error, particularly when the numbers you provide are going to inform a further decision process. (My students that read my blog are rolling their eyes right now because they are finding out I write exactly what I say in class.)

If you look at the Rev-e-News documents provided by the state OMB you will see a forecast value provided, and the date is usually at some point in the legislative session. That date stays the same over the entire two-year period. It appears it was never updated. North Dakota gets a forecast for revenues during the legislative session and then seems to let it ride. As an example, here is a link to the Rev-E-News for July of 2015. It provides data for revenue collections in June of 2015. Notice on page 2 there is revenue data for the “fiscal month” and the “biennium-to-date”, both forecast values and actual. Even in June of 2015 they used the legislative forecast for May of 2013.

This is horrible practice. Regardless of what the underlying model is, the variables chosen and the methodology adopted, you are essentially ignoring updated information. If the forecast was wrong, why? Were there shifts in underlying variables such as employment or oil price? Were there weather events that kept people home instead of shopping? Every month we get updated revenue numbers but there is no update of the forecast. Not to put too fine a point on this, but any of my students would likely attest that if they did that with an assignment or project in my class, they would fail the project. This is true of misses where the forecast is too low, as well as when the forecast is too high. When you do not update when given new information you really have no hope of being accurate.

Now before I demonize too much, I will say I do not know the structure of the contracts and the payments made. This may be what the state pays for and is properly priced according to the provider’s fee schedule. I do not have that information, but that does not make it good practice, and we probably need to at least add the cost of the special session to what that forecast ultimately cost the state of North Dakota.

This work by Flynnglis is licensed under a Creative Commons Attribution-NonCommercial 4.0 International