Why is it impossible to find a Good Forecast Model?

1. MYTH #1: Aggregation or Disaggregation has no impact on variation.

At a store, an item may sell between 10 to 100 pieces (10x Variation)
For manufacturer, the difference might be 1,000 to 2,000 pieces (2x Variation).

The more disaggregated the data is, the higher the variation.

2. MYTH #2: Sophisticated algorithms can solve all problems.

If I can sell, between 10 to 100 pieces of an item in a store and if I have 10 stores, how many pieces do I need?

The average pieces required in a store is 55 pieces. So if I give 55 pieces in all 10 stores, will I have myself covered?

Never!

In some stores, you will end up selling less than 55 pieces. You will have excess stock in the stores. 
In other stores, you will end up selling all 55 pieces. You will have shortage in those stores.

So, will we be better if we keep, say 70 pieces in each store?

Nope.

Forecasting algorithms are getting more and more complex. Companies need to justify to the client that the new version will bring “better” results this time. 

3. MYTH #3: Forecast will take care of Sudden changes.

Toilet paper disappearing from shelves at whiff of a calamity.
Music albums stockout for recently died musician.
100x increase in sales of a food item which has been discovered to be a ‘Superfood’ .

The more sudden the change in demand, the worse the forecast is.

In today’s world of sudden changes, relying on granular forecast is a recipe for disaster.

The advanced forecasting modules try to come up with an answer to the below question:

What product to hold at which place (where) and when. 

Notice that this puzzle has three questions: 
– what, 
– where, and 
– when. 

To be a good forecast of demand, forecasting has to answer each of these questions. 

If you want to use forecast for the most granular level, you are in for trouble.