January 14, 2025 · 2 min read
What a forecast can and can't tell you
A forecast gives you a range of likely outcomes if the past keeps holding, and it goes quiet when the process changes.
We build forecasting software, so we often get the question of what a forecast really does. The short answer: a forecast is a probability distribution over possible futures. It does not point at one number and say “this is what happens.” It says “here is the range of outcomes that look likely, and here is roughly how sure you can be about that range.” All of this rests on one quiet assumption: that the patterns in your past data keep holding.
That assumption is where forecasts do their work, and where they break.
A model learns from the past. Things behaved a certain way before, so they will probably behave that way again. When this comparison with the past holds, it works well and it is cheap. When the process underneath changes, the model cannot know. It keeps reasoning from yesterday’s world, and it does so with the same calm confidence it had when it was right.
Take a demand model trained on five years of retail sales. Inside those numbers is a picture of how customers shop, how often, and in what quantities. Change that behaviour overnight, for example a channel disappears or a new habit forms, and the data now describes a world that no longer exists. The math is fine. The arithmetic is correct. The assumption is old.
So what do you do with it
Trust the forecast for what it is good at. It puts a number on uncertainty instead of hiding it, and that alone is better than a single guess dressed up as a plan. What it cannot do is tell you whether the ground is about to move. That call needs domain knowledge. It needs the messy intuition of people who know the business and have seen it change before. This is the kind of judgement you can’t put into training data.
The best way we have seen teams work is to treat the model as a first opinion, a baseline worth arguing against. Not “what does the model say, let’s go,” but “what does it say, where do we think it is wrong, and why.” This is slower, and it needs people who understand both the numbers and the company. It is also the version that holds up. The machine works through history without getting tired. The person notices when history stopped applying. Keep both in the room.