ND Sales Tax Forecast Performance

This is going to get a bit more technical than many of my other posts. However, I am a big believer that there is no reason to shy away from complexity, particularly when avoiding it sacrifices accuracy. So we are going to discuss forecast performance for sales tax in North Dakota. There are many different ways to evaluate forecasts and the one I will use here is called a tracking signal.

The tracking signal is a way to see how far away, plus or minus, the forecast is from reality. Now you might not think this appropriate because the consequences of the forecast coming in below actual are distinct from the consequences of coming in above actual. This is a fair point, but accuracy is accuracy at some level and most other measures would tell us the same thing.

The formula for tracking signal is:


where MAD is the mean absolute deviation. The general rule here is if the value of the tracking signal gets above four or below negative four (the limits here are a bit flexible) you need to revisit your forecast model because the errors are getting too large.

So let’s look at the tracking signal for North Dakota:


Fully 48% of the time the tracking signal is outside the boundaries I established. The forecast models were not working well. Obviously we cared less about this when the misses were underestimates of revenues. The interesting thing is that in the last biennium we were closer and stayed within the bounds. By the way, for the first six months of the current biennium you can see it was bad and getting worse.

We need better forecasts to give us a better understanding of the revenue generation in the state. This should allow for better planning, (sigh) or some planning, in a forward-looking (sigh) fashion.

5 Comments, RSS

  1. Ryan Jockers August 4, 2016 @ 7:04 am

    Makes me wonder what their technique is for forecasting sales and use taxes, whatever it is, they might need to modify it quickly.

    • David August 4, 2016 @ 8:20 am

      Modify is too polite. This is one of those situations where you need to through the whole thing out. Part of the issue is multiple groups had the contract over this period and so it seems likely the models were drastically different. My guess is right now there is a standard state level model with some parameters specific to North Dakota, but that likely does not capture enough with oil sector volatility. We are not Oklahoma or Texas.

      • Ryan August 4, 2016 @ 8:54 am

        Agreed, it is very surprising to me that they haven’t incorporated at least some direct and/or indirect indicators of oil production/prices into their model that are specific to North Dakota. What’s interesting is that it didn’t become obvious until oil prices started to drop.

        It’s also good to note that it is not as if the actual revenue would change if their forecasts were more accurate. It would, however, give the state agencies at least a chance to plan and adjust their budget accordingly without having to cut 2-3 times a biennium. This especially makes capital appropriations and planning a nightmare.

        The most damaging part about the forecast being off so much is the lack of trust. Unless the OMB makes it clear they are revising their methods and taking action to redo their model, who is going to trust their forecasts anymore?

        • David August 4, 2016 @ 11:58 am

          Love your comments here. Spot on.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.