This is going to get a bit more technical than many of my other posts. However, I am a big believer that there is no reason to shy away from complexity, particularly when avoiding it sacrifices accuracy. So we are going to discuss forecast performance for sales tax in North Dakota. There are many different ways to evaluate forecasts and the one I will use here is called a tracking signal.

The tracking signal is a way to see how far away, plus or minus, the forecast is from reality. Now you might not think this appropriate because the consequences of the forecast coming in below actual are distinct from the consequences of coming in above actual. This is a fair point, but accuracy is accuracy at some level and most other measures would tell us the same thing.

The formula for tracking signal is:

where MAD is the mean absolute deviation. The general rule here is if the value of the tracking signal gets above four or below negative four (the limits here are a bit flexible) you need to revisit your forecast model because the errors are getting too large.

So let’s look at the tracking signal for North Dakota:

Fully 48% of the time the tracking signal is outside the boundaries I established. The forecast models were not working well. Obviously we cared less about this when the misses were underestimates of revenues. The interesting thing is that in the last biennium we were closer and stayed within the bounds. By the way, for the first six months of the current biennium you can see it was bad and getting worse.

We need better forecasts to give us a better understanding of the revenue generation in the state. This should allow for better planning, (sigh) or some planning, in a forward-looking (sigh) fashion.

This work by Flynnglis is licensed under a Creative Commons Attribution-NonCommercial 4.0 International