Backtesting Systemic Risk Forecasts using Multi-Objective Elicitability

Backtesting risk measure forecasts requires identifiability (for model calibration and validation) and elicitability (for model comparison). We show that the three widely-used systemic risk measures conditional value-at-risk (CoVaR), conditional expected shortfall (CoES) and marginal expected shortfall (MES), which measure the risk of a position Y given that a reference position X is in distress, fail to be identifiable and elicitable on their own. As a remedy, we establish the joint identifiability of CoVaR, MES and (CoVaR, CoES) together with the value-at-risk (VaR) of the reference position X. While this resembles the situation of the classical risk measures expected shortfall (ES) and VaR concerning identifiability, a joint elicitability result fails. Therefore, we introduce a completely novel notion of multivariate scoring functions equipped with some order, which are therefore called multi-objective scores. We introduce and investigate corresponding notions of multi-objective elicitability, which may prove beneficial in various applications beyond finance. In particular, we prove that conditional elicitability of two functionals implies joint multi-objective elicitability with respect to the lexicographic order on ℝ2, which makes it applicable in the context of CoVaR, MES or (CoVaR, CoES), together with VaR. We describe corresponding comparative backtests of Diebold-Mariano type, for two-sided and 'one and a half'-sided hypotheses, which respect the particularities of the lexicographic order and which can be used in a regulatory setting. We demonstrate the viability of these backtesting approaches in simulations and in an empirical application to DAX 30 and S&P 500 returns.