Backtesting VaR models: Quantitative and Qualitative Tests
Carlos Blanco and Maksim Oks
This is the first article in a two-part series analyzing the accuracy of risk measurement models. In this first article, we will present an overview of backtesting methods and point out the importance of conducting regular backtests on the risk models being used. In the second article, we will present an alternative to measuring VaR using a top-down or “macro” approach as a complementary tool to traditional risk methodologies.
Should risk models be accurate?
Firms that use VaR as a risk disclosure or risk management tool are facing growing pressure from internal and external parties such as senior management, regulators, auditors, investors, …show more content…
A more rigorous way to perform the backtesting analysis is to determine the accuracy of the model predicting both the frequency and the size of expected losses. Backtesting Expected Tail Loss (ETL) or Expected Tail Gain (ETG) numbers can provide an indication of how well the model captures the size of the expected loss (gain) beyond VaR, and therefore can enhance the quality of the backtesting procedure.
* If we do not have ETL, VaR(+) and ETG data, we can perform the analysis with VaR data exclusively, but we would have limited information to extract conclusions.
Quantitative tests:
Statistical tests help us check whether the risk model is accurately capturing the frequency, independence or magnitude of exceptions, which are defined as losses (gains) exceeding the VaR estimate for that period.
When we test a certain hypothesis in statistics, we can make two types of errors: Type I errors occur when we reject the model which is correct, while type II errors occur when we fail to reject (that is incorrectly accept) the wrong model. It is clear that in risk management, it can be much