The current world-wide credit crisis has triggered a much needed examination of risk management technology to determine which techniques worked and which did not. Value at Risk technology ("VAR") is probably the first risk management technique to be tied to a specific firm. In fact, the J.P. Morgan "brand" probably did more than anything else to popularize VAR. VAR, however, has not fared as well in the credit crisis as J.P. Morgan Chase has. This post tells why.
VAR has as many incarnations as there are risk managers. In the mid-1990s, J.P. Morgan made the decision to publicize the technique and to spin off an affiliate that would resell the VAR technology and related information. Hundreds of financial institutions adopted the VAR technology as a supplement for or replacement of traditional trading floor-based risk limits like "PVBP" (present value of a basis point move in interest rates) or "delta" type measures with respect to other risk factors, like stock prices and foreign exchange rates. Philippe Jorion has done a masterful job in his book at documenting various ways to implement the VAR concept. The purpose of this note is much simpler--to answer the question "Does it work?" Could a sophisticated user of the VAR concept have anticipated the current crisis and taken action early on to mitigate the impact of the crisis? The answer is...maybe. That's not a sufficiently strong endorsement to justify the considerable sums that many institutions have invested in legacy VAR technology. Why is the answer so lukewarm? We turn now to the reasons.
First of all, there are three principal variations of VAR technology, and they have different strengths and weaknesses:
A. Historical VAR, which estimates the nth percentile in potential losses based on historical returns on the relevant traded assets.
B. Variance-Covariance VAR, which takes the historical volatility and correlation of returns on traded assets to calculate the nth percentile in potential losses
C. Monte carlo VAR, which simulates the environment forward, rather than using historical data like A and B.
Most of the initial publicity around the VAR concept focused on a 10-day time horizon for the calculation, since it was advertised as an index of short term trading risk for assets traded in an active market. Most implementations assume that the portfolio of assets and liabilities that one is analyzing stays constant over the one period horizon (with length 10 days).
How well did these techniques serve their users in the credit crisis? On January 28, 2008, a story on bloomberg.com entitled "Death of VAR Evoked as Risk Taking Vim Meets Taleb's Black Swan" reported that Merrill Lynch's historical VAR measure was $92 million at a time that CDO losses had already reached $18 billion at Merrill. With all due respect to Nassim Taleb, this was no "black swan." The error in the VAR measure at Merrill was due to simplifying assumptions in the calculation that were simply not true, leading to incorrect risk assessment. With respect to historical VAR, here are the major problems that would lead one to underestimate risk by a factor of 200 like Merrill did:
Problem 1: Future returns almost never equal historical returns. Home prices dropped, mortgage default rates sky rocketed, and the historical returns (which are typically measured over a period of less than a year) did not adequately reflect the possibility of these events
Problem 2: Much of the losses in the current crisis have stemmed from assets that either always have been or are currently "non traded": subprime mortgage loans, Alt-A loans, and structured products using them as reference collateral. They were simply "left out' of the calculation
Problem 3: The balance sheet you end with is not the balance sheet you start with, even though that's the assumption made by most VAR users. One of my favorite risk managers, the Wizard, works at one of the world's largest life insurance companies. He once commented, "Don, this VAR concept doesn't make any sense. If we extend the time horizon to a year, a huge percentage of our assets would have matured by then. How could anyone assume a constant balance sheet for VAR?" As usual with the Wizard, his comments were right on the mark.
Most of these problems apply to variance-covariance VAR as well, with an additional monkey wrench thrown into the gears:
Problem 4: Returns on most securities are NOT normally distributed. In our post on the biggest problems in FAS 157 (updated April 2, 2009), we noted that if one assumed that the returns on Bear Stearns common stock were normally distributed at their historical mean and variance, one would have implied a probability of failure (a monthly return of -100%) of 0.000000. For CDO tranches with narrow bands of losses, the outcome is almost exclusively "all or nothing," since it becomes highly unlikely for the tranche to suffer partial losses when the size of the tranche is a small percentage of total notional principal of the collateral underlying the CDO. The net impact of the normality assumption is a dramatic underestimation of risk.
When one turns to monte carlo VAR, still another problem can be important:
Problem 5: Returns in period N are not independent of the returns in period N-1, even though this assumption of independence is very, very common in financial theory and practice. Take the Case-Shiller home price index from www.sandp.com and test the hypothesis that home price returns are independent of returns on prior periods to see for yourself--the returns are very, very highly cyclical and dependent on past returns.
Saving Value at Risk
Can VAR be saved? The answer is "yes" if the analyst is very careful and has a powerful enterprise wide software package like Kamakura Risk Manager. Here are the steps that one needs to take to get maximum accuracy from the VAR way of thinking:
Fix Number 1: Do the VAR calculation on a multi-period basis that allows for a dynamic balance sheet, with cash flow reinvested, new assets and liabilities generated, and embedded options exercised
Fix Number 2: Allow for defaults to occur, since they are the biggest source of "non-normality" in the real world. Kamakura Risk Manager uses default probabilities from the Kamakura Risk Information Services default service (or any other source) for this purpose
Fix Number 3: Recognize that the drivers of risk, typically macro economic factors, can be cyclical with probability distributions in period N that depend on prior periods.
Fix Number 4: Look at the entire distribution of value, not just one percentile level, at multiple points in time
Fix Number 5: Stress test VAR and mark to market value with respect to all relevant macro factors (like home prices) to avoid being lulled into complacency. This requires explicit links between the macro factors, default probabilities, and credit spreads.
With these fixes, a multi-period default-adjusted VAR calculation can add some insight that creates incremental risk-adjusted shareholder value added.
Donald R. van Deventer
Kamakura Corporation
Honolulu, April 6, 2009