Combining Models: Ensembles in ViEWS
Every month, ViEWS estimates a number of constituent models that forecast 36 months into the future. To make full use of the information in these constituent models, ViEWS combines them into ensembles. Research has shown that ensembles improve the reliability and accuracy of forecasts (Armstrong, 2001). Since they incorporate the strengths of multiple constituent models, ensembles are often said to contain the "wisdom of the crowds''.
ViEWS currently estimates two forms of ensembles: unweighted average ensembles and Ensemble Bayesian Model Averaging (EBMA, Montgomery, Hollenbach & Ward, 2012). In the unweighted average ensemble framework, each (calibrated) constituent model is assumed to be equally informative. However, this may not always be a plausible assumption. In the EBMA framework, constituent models are therefore weighted according to out-of-sample predictive performance. In this way, the EBMA produces ensembles that give greater weight to the most informative models and models that pick up unique variation. While ViEWS estimates both ensembles, the current official forecasts are based on the unweighted average ensembles.
To see the combination of models in ViEWS more clearly, consider the predictions of state-based conflict events from three constituent models in Figure 1. The color schema on the right hand side indicates the predicted probability of state-based conflict. The three models overall predict conflict in the same geographic areas, but the the spread of the areas differ. This is because the models displayed in the center and on the right side incorporate factors at the country-level, whereas model displayed on the left ignores country-level factors. The country-level influence is especially apparent on the right side, where conflict risk largely follows country borders.
Each of the three models have merit, and in combining them, we can obtain better forecasts. In Figure 2, we can see the predictions from an ensemble than contains the three constituent models in Figure 1 as well as a number of other constituent models. The ensemble models differs in important ways from each of the constituent models. For example, comparing the ensemble in Figure 2 to the forecast on the right side in Figure 1, we can see that the forecasted risk of conflict is lower overall. There are especially big differences in the South of Algeria and North-West of Mali, where the probability of conflict is pulled down from 1-5\% to around 0\%. Moreover, we can see that the predicted probability of conflict in Madagascar is a compromise between the models displayed in Figure 1. The models displayed on the left and in the center predicts higher probability of conflict in Madagascar than the model on the right. In Figure 2, we can see that the predicted probability is low for most of Madagascar, but that some areas are assigned higher probabilities. A similar compromise is apparent in countries such as Sierre Leone, Ivory Coast, and Liberia in the North-West.
- Figure 1: model with indicators at PGM level only
- Figure 1: model with indicators at PGM and CM level
- Figure 1: multiplied forecasts from one model fitted at the PGM level and one model fitted at the CM level
- Figure 2: forecasts of state-based violence from an ensemble of constituent models
Please cite: Hegre, Håvard, Marie Allansson, Matthias Basedau, Michael Colaresi, Mihai Croicu, Hanne Fjelde, Frederick Hoyles, Lisa Hultman, Stina Högbladh, Naima Mouhleb, Sayeed Auwn Muhammad, Desiree Nilsson, Håvard Mokleiv Nygård, Gudlaug Olafsdottir, Kristina Petrova, David Randahl, Espen Geelmuyden Rød, Nina von Uexkull, Jonas Vestby (2019) ‘ViEWS: A political violence early-warning system’, Journal of Peace Research, 56(2), pp. 155–174. doi: 10.1177/0022343319823860.
- Armstrong, J. S. (Ed.). (2001). Principles of Forecasting: A Handbook for Researchers and Practitioners (Vol. 30). Springer Science & Business Media.
- Montgomery, J. M., Hollenbach, F. M., & Ward, M. D. (2012). Improving predictions using ensemble Bayesian model averaging. Political Analysis, 20(3), 271-291.