7 Forecast evaluation
7.1 Evaluating forecasting systems: Scoring rules
Forecast evaluation involves analyzing how well your forecasting system generates predictions that match observations.
Elements of a forecast evaluation:
Data: a sequence of matched ordered pairs of forecasts + corresponding observations
A scoring rule: A formula that maps the sequence of forecasts and observations to a summary metric of the quality of the forecasting system
Important issues:
Measurement error in observations: In some cases, verifying observations may not be known with perfect precision.
*Scoring rules
7.2 Q: “How good is your forecasting system?”
How would you answer?
Different ways to measure forecast quality:
- Correlation between forecasts and verifying observations
- …