Demand Metrics for Forecast Diagnostics

We just published a new Demand Metrics Diagnostics Template for all DPNet users.

This template is provided as a reference to calculate the health of the modeled forecast for one Product/SKU over time.  There are a variety of metrics provided by both academics and software providers causing a lot of confusion about what each of these mean.  Hopefully, this template will settle the debate by clarifying each metric and illustrating the correct method to calculate these metrics.

This template can be used as a model diagnostic to evaluate the fitted model.  There are several metrics available to use as a model diagnostic including R-squared, Running Mean Absolute Deviation, Weighted Mean Absolute Percent Error, Co-efficient of Variation, Mean Absolute Scaled Error, Geometric Mean Absolute Relative Error and the Forecasting Efficiency Quotient.  You can also track forecast bias using the Forecast Bias measure, the Tracking Signal and the Mean Percent Error.

This template can also be used to calculate the observed forecast error for the same SKU if you have a history of Lag forecasts available for the same SKU.  This type of reporting is made available in our Vinayware software package which tracks multiple versions of the forecast.

The template is provided for just a few months.  If you want to calculate this over many months, insert rows and the formulas will self-adjust.

Based on our consulting experience we believe, Operation Management Professionals using these metrics will be able to judge the health of their forecast and understand their demand plans in a better way.

There are a couple of new metrics worth mentioning:

1.  Geometric Mean Absolute Relative Error – Calculate how improved your forecast is compared to a naive forecast.  This is a complex and esoteric measure – it compares the improvement in forecast accuracy compared to a naive model that uses either an average or a moving average or a policy of using the last month actual as the forecast.

2.  Forecasting Efficiency Quotient – We are introducing this new metric as an alternative to the GMRAE and something of an easier measure to understand by the businesses.

More on the FEQ in a separate blog entry.  Stay Tuned!

Now you can download this new metrics template at http://demandplanning.net/templates.htm or at this other link

http://demandplanning.net/DemandMetricsExcelTemp.htm.