The Golden Rule of Forecasting is, “be conservative” when forecasting by relying on cummulative knowledge about the situation and about forecasting. The working paper by J. Scott Armstrong, Kesten C. Green, and Andreas Graefe is available here.
This experiment provides access to the online tool Golden Rule for Forecasting Checklist software.
Purpose and instructions:
1. Even if you are unfamiliar with forecasting, you can use the Golden Rule Checklist to determine whether forecasts are independent and were derived using the best available forecasting methods.
2. If you are using the Golden Rule Checklist software for the first time, you will need an hour or two to become familiar with the software, with the descriptions of the Checklist items, and with the ratings procedure. Reading the Golden Rule of Forecasting: Be Conservative is good preparation.
3. Unless you are already familiar with the forecasting procedures, understanding how the forecasts were derived is the most time-consuming preparation you will need to do. Once you have done that, you should be able to rate the forecasting procedures in less than an hour (if you have already read the paper mentioned in #2 above).
4. Rate the forecasting procedures against all relevant Checklist items. You may need to ask the forecasters for more details on their procedures. Click on a Checklist item header for a detailed description. Use the comments column to record questions.
5. Obtain independent ratings of the forecasting procedures from two or more people. The Golden Rule Checklist software can summarize ratings for any number of raters, provided that they use the same name for the rating task.