Value of judgemental adjustments to promotional forecasting
Sales forecasting is becoming increasingly complex, due to a range of factors, such as the shortening of product life cycles, increasingly competitive markets, and aggressive marketing. Companies often use some form of Forecasting Support System (FSS) to integrate univariate statistical baseline forecasts with expert judgement from demand planners, essentially adding information from additional factors to the statistical forecasts. Promotional and advertising activity is one of the main reasons for such adjustments.
Alternatively, statistically adept forecasters could follow the route of building a model that includes promotional information. Although such models can be of varying complexity, it is reasonable to expect companies to be able to use simple regression type models. A valid question then is, do human experts still add value? Is there still a benefit of judgementally adjustmenting the demand forecasts?
The work by Trapero et al. (2013) attempted to investigate this question. They looked at a case study of an FMCG company that typically adjusted judgementally exponential smoothing forecasts to cater for promotions. They looked at three different forecasts:
- Statistical Forecast (SF) – provided by the company
- Final (Adjusted) Forecast (FF) – provided by the company
- Transfer Function Forecast (TF) – incorporating an indicator for promotions
Fig. 1 presents the forecast error for different sizes of negative and positive adjustments during promotional periods. It also provides the frequency of adjustment for each case.
As expected the statistical model that includes promotional information (TF) is consistently better than the baseline univariate statistical forecast (SF). However, that is not true when comparing the adjusted final forecast (FF) with TF. There is a range of positive adjustments that human experts incorporate beneficial information, resulting in lower forecast errors. This is not true for very large adjustments, where experts perform even worse than the univariate forecasts.
Fig. 2 presents the results during non-promotional periods. Both FF and TF are more accurate for negative adjustments. This is essentially because both are lowering the forecasts appropriately when a promotion is finished, while the univariate model does not. Again, human experts add value for small and moderate positive adjustment, resulting in better accuracy than both statistical models.
So according to the analysis of Trapero et al. there are benefits of expert judgement even when promotional information is included in statistical models (albeit in a rather simple one). This is presumably because they take into consideration additional soft information, which may not always be easy to include in statistical models.
A reasonable follow-up question is: does it make sense to combine the predictions of TF and FF, so as to use all available information? In the same paper this analysis is carried out and the finding is that such a combined model performs consistently best. However, as the authors point out, this assumes that experts will keep on adjusting in the same way even when they know that their forecast will not be the final one. To me that seems to be a leap of faith and this is a question that should be researched further.