How Does Forecast Accuracy Matter Anyway?

I recently saw a post by Sujit Singh on LinkedIn that tabulated how one would rate different companies on their Forecasting process based on observed Forecast Accuracy levels. The below table is slightly adapted from his classifications - ranging from Excellent (top of the chart) to Very Good all the way down to Very Poor (or nearly Hopeless).

The question is, which companies can be helped to move up the ladder into the best in class and which companies are hopeless and will be stuck with their bad demand forecasts despite investment in expensive ERP and Planning tools.

Is the best in class reachable only to those in the Mediocre/Average and the Very good segment?

My own experience working with many of the Valtitude consulting clients is this - NOT just an opinion - those in the poor segment can see substantial improvements in their demand forecasting process and a big jump in the observed forecast accuracies with a bit of effort in three things - process, tool re-engineering, and training. We have proved this again and again with many of our clients striving to transform their Demand Planning and S&OP.

Of course, this post is not for those who neither believe in forecast accuracy nor in measuring or creating point forecasts for Supply Chain Planning and IBP. Forecast accuracy for the purpose of this discussion is measured as the complement of the Forecast error based on a point forecast - regardless of whether this qualifies and meets your criteria - we are neither talking "interval forecasts" nor "probabilistic forecasts" here.

Repeatedly our best turnaround has been on the clients who have belonged in the class 25% - 50%. These companies believe that demand visibility into the future and a clear demand signal to the supply chain will go a long way in optimizing their inventory levels, provide better visibility to their supply partners and create efficiencies which in turn may translate into better vendor pricing and attractive purchase terms. Long-term visibility at some aggregate level can also result in avoiding the capacity bull-whip or the boom-bust scenarios.

With diligent effort across the board, we have been able to lift this group to 70+ and even over the 85% threshold. What does this diligent effort require?

The planning re-innovation effort typically includes a few steps -

• Resolving simple Master data inconsistencies - though many client personnel knows there are issues, they are unable to address this due to the lack of analytical horsepower or simply lack of time away from their day jobs.
• Refining the planning process and resolving the frictions in the cross-functional touchpoints. At times, this may have implications for the underlying business process as well. Addressing the process issues may be easier but expanding this to get cross-functional adoption may be much more difficult. It is even harder if it points to changing the business process or business model. For example, SKUs with extra-ordinarily intermittent demand also have low margins or negative gross margins. Telling the business to weed out such SKUs may not be a popular idea!
• Understanding the purpose of forecasting and identifying where NOT to use forecast and mess up the supply deployment process
• Implement or re-implement the process/tool to create a good Statistical baseline forecast - a bigger initiative than just the process and may involve the CIO to open his budget.
• Revamp the organization to hire the right skill sets
• Training on the process and the tool - without training nothing will move. After a few iterations, the planning community will go back to the original point and of course - Office 365 spreadsheets.
• Reinforcement on a monthly basis to both evaluate forecast quality and support the Demand Consensus process

I would say this group (25-50) is the sweet spot - because so many simple things are messed up. It would be amazing to the external eye how much basic stuff is missing - how common sense becomes a rare commodity when you are back is up against the wall with every day "busy" stuff.

How simple? You may even typically see a straight line forecast in the system when the data shows a clear pattern because the planning team has time only for an annual forecast that is split equally.

No one noticed that the planners didn't know how to set up a graphical view in the system and no one looked at the entire time series as a graph. They only focused on the next month or the next two months and manually re-created the seasonality. Only some clients have the willingness to adapt to change and produce great results.

Next, onto the Very Poor group which has less than 25% forecast accuracy. Interestingly, there may be very few that will be here as typically these are companies that do not measure forecast accuracy. They think they are good so they don't have to measure. Some measure at the wrong level and pat themselves on the back.

A few just don't want to measure given the culture or management style or other pressures - unfortunately, we cannot help them.

Interestingly, their excuse is "Oh our business is complex and hard to forecast!" It is a pity they will stay that way for a long time.

If you want to move up the ladder, the first thing you do is to measure the quality of your demand signal honestly at the detail level or the level of deployment. If you have difficulty measuring this or do not know how, give us a call.