The Federal Reserve’s 2015 Comprehensive Capital Analysis and Review stress testing exercise is moving toward its conclusion, but the biggest banks in the United States live in fear of delivering a “wrong answer” to the Federal Reserve. None of the big banks is immune from this fear, including the big four Bank of America (BAC), Citigroup (C), JPMorgan Chase (JPM), and Wells Fargo (WFC). In this note, we explain this fear and present a worked example of the right answer, a stress test using the Federal Reserve’s CCAR 2015 scenarios for an equal-weighted portfolio of loans to all public firms in North America. Along the way, we make suggestions about how to improve the CCAR stress testing process from the point of view of both the Federal Reserve and the banks that it regulates.
Background for Stress Testing Credit Losses to Public Firms
One of the fears bankers have is presenting stress test results to the Federal Reserve that contain mistakes, leading the Federal Reserve to reject the results. In recent notes, we have outlined a number of common errors that rightfully should lead to the conclusion that the bank’s stress test relied upon invalid models. We list a few common errors here:
An Aversion to Using Models that are “Too Good”
Another fear that some bankers have is that superbly calculated stress test results backed by best practice econometrics and documentation will be incorrectly rejected by the Fed. This can happen for two reasons. The first is a lack of understanding by the front line regulatory examiners of those calculations. The second is the tyranny of the “severely adverse scenario” in the CCAR process. We discuss that in the next section.
Returning to the first point, there is anecdotal evidence that some of the biggest and most sophisticated banks in the United States are pulling their punches, so to speak, from an analytical point of view. Instead of using their most sophisticated staff members and best credit models, there is anecdotal evidence that they have instead intentionally employed inferior models because they believe these inferior models are more likely to gain approval. How do they know which inferior models to use, given that their own history with the CCAR process is so short? They engage the usual set of consulting firms, expert in transporting the lowest common denominator analytics across the financial services industry, and use nothing more sophisticated than these lower common denominator models. This is surely not what the Federal Reserve and the U.S. Congress intended to happen.
An Aversion to Producing Results That are not “Severely Adverse”
Many bankers tell me that their stress test scenarios must meet a very important political constraint. The “adverse” scenario must show losses that are greater than in the “base line” scenario, and the “severely adverse” scenario must produce losses that are greater than the “adverse” scenario. These bankers, most of whom have many decades of risk management and bank political experience, are convinced that there is no chance of stress test approval, no matter how well documented and presented, for a calculation that does not meet these constraints. If this fear is justified, it is an embarrassment to U.S. bank regulators and to the U.S. Congress that has given them their mandate. We explain why with a few examples.
Let’s assume that a bank subject to CCAR has successfully hedged or securitized all of its credit risk, so interest rate risk is its only concern. Let’s look at the base line, adverse, and severely adverse scenarios for the 3 month U.S. Treasury bill rate, one of the Fed’s most important CCAR macro factors. We show historical movements and the scenario forecasts in this graph:
The graph shows the actual evolution of the 3 month Treasury bill rate since 1975 in black. The base line scenario given by the Federal Reserve is shown in green. The adverse scenario is shown in blue, and the severely adverse scenario is shown in red. We have no quibbles with the scenarios per se. We are simply interested in their implications. Relative to the base line scenario, the 3 month T-bill rate is higher in the adverse scenario (in blue) and lower in the severely adverse scenario (in red). That’s right—interest rates move in the opposite direction in the two adverse scenarios. If the bank’s only risk is from a rise in short term rates, it will indeed have worse results in the adverse scenario than in the base line scenario. Even more concerning, the severely adverse scenario will be the best scenario from an interest rate risk point of view. But the bankers I talk to are afraid to present this result to the Federal Reserve.
The same relative movements, compared to the base line scenario, can be seen for the 5 year U.S. Treasury yield in this graph:
Again, if interest rate exposure is to the five year point on the yield curve, a bank’s stress test results should show that the adverse scenario is the worst outcome and the severely adverse scenario is the best scenario. The same opposite movements are found with the U.S. Treasury 10 year yield and the U.S. fixed rate mortgage yield.
What if the bank has swapped all of its basis risk related to the prime rate so that the bank earns a fixed rate on the swap and pays the prime rate, a plausible transaction in a “reach for yield” environment. The graph below shows the Fed’s scenarios for the prime rate.
The prime rate will be highest in the adverse scenario, hurting the bank. The base line scenario will not be as bad. The severely adverse scenario will again be the best scenario, but again bankers are afraid to present this result to the Federal Reserve.
What about exposure to credit spread movements? Let’s assume that the bank has fully hedged all of its default risk (for simplicity we assume the payoff in the event of default comes at the scheduled maturity of the credit, not before) with a riskless counterparty but still has credit spread exposure. We also assume the bank has an advantage in asset generation so that the hedged credit spread is not reduced to zero. If the bank is the owner of hedged credit risk, it will gain if spreads contract and lose if spreads widen. And yet look what the Fed’s BBB credit spread scenario calls for:
Spreads are widest in the adverse scenario, hurting the bank the most. By the end of the 13 quarter forward simulation, credit spreads in the severely adverse scenario are below credit spreads in the baseline scenario. Again, the severely adverse scenario is the best outcome for the bank, and the adverse scenario is the worst. We remind the reader that we have no qualms about the scenarios themselves. Our concern is the bankers’ concern: they believe they cannot present an outcome from the stress-testing process that is not ordered from best to worse in this order: base line, adverse, and severely adverse.
We now put aside this political dilemma and concentrate on a worked example, stress testing losses on a portfolio of exposures to one asset class. Because, by definition, the information on public firms is public, we use that asset class to measure the relative impact of the base line, adverse, and severely adverse scenarios.
Stress Testing: A Worked Example of Public Firm Credit Exposure
Best practice stress testing for a public firm-related credit exposure involves the following steps:
- Secure a high quality default probability term structure for each firm with as much history as possible
- Avoid common modeling mistakes as outlined in our note on the use and abuse of lagged default probabilities in credit models , a practice described as a “forbidden model” by Angrist and Pischke (2009).
- Using proper recognition of the correct lags in known values of company specific input variables, construct the most accurate econometric relationships linking macro factors and company default probabilities. Stock and Watson (2007, pages 646-647) described the “direct multi-period forecasting” process and related econometric procedures to avoid common errors.
- Benchmark valuation and cash flow software as described in a recent note so that mark to market of current credit exposures to the public firms in the portfolio produces correct time zero market values for those credits.
- Apply the Federal Reserve scenarios and produce proper valuations, cash flows, accounting net income, and credit losses for each firm.
- Aggregate to get portfolio results and combine with other asset classes for final CCAR reporting.
The results will vary by the identities of the firms in the portfolio and the amount and nature of the credit exposure. In order to quickly evaluate the relative severity of losses under the base line, adverse, and severely adverse scenarios, we use “common practice,” not “best practice,” in fitting the relationships in step 3. We fit macro factors to portfolio losses to a “historical loan portfolio,” something many banks in the CCAR process have been forced to do when their own data archives do not allow name-specific and transaction specific analysis over a long period of time. Fitting macro factors to portfolio losses is common practice, not best practice, for these reasons:
- Fitting portfolio losses is dependent on the dollar distribution of exposures to each name over the historical time period. Forward looking exposures will certainly be different.
- The nature of exposures is assumed to be constant, but it is not.
- The pool of reference names is assumed to be the same as the historical pool, but it is not. By using historical losses, one is essential sampling “with replacement.” In a real pool of names in which individual names default, there is no replacement and the average quality of the pool improves. This is exactly what has happened since the credit crisis in the U.S. mortgage market, but we assume this impact away. In effect, we assume the bank does not learn from bad loans and makes the same loans to the same borrowers again and again. Most bankers we talk to believe that this is what the Federal Reserve requires, even though it is not the most accurate “best practice” approach.
Our hypothetical portfolio at March 31, 2014 is assumed to have these characteristics:
- All credit exposures have an identical nature and identical dollar amount.
- The reference names in the portfolio are all listed firms in the United States and Canada on March 31, 2014 for which both stock price information and financial statement were available. The total number of names in the portfolio at that time was 8,015 firms.
We fit historical loss rates on this pool of all public firms in North America back to 1990. Using all of the Fed’s domestic CCAR macro factors and transformations of them, we fit a relationship to the loss rate on this pool over 96 quarters. Fifteen explanatory variables were statistically significant. Using a transformation of the default rate on the pool that ensures a projected default probability in the range from 0% to 100%, we found an adjusted r-squared of 85.04%. The stress test results are shown below.
The results are consistent with the bankers’ hope to present results to the Federal Reserve which are ordered from best to worst as desired: the base line losses are the smallest, the adverse losses are next, and the severely distressed losses are the worst. The severely adverse scenario shows that losses surge quickly higher due to the assumed rise in volatility and fall in commercial real estate prices, among other factors. At their peak, losses in the severely adverse scenario show a default rate of less than 3%, well below the peak in excess of 6% in the credit crisis. By the midpoint of the 13 quarter simulation, losses on the severely adverse scenario fall below the adverse scenario losses and, at the very end, below the base line scenario.
We have demonstrated that a best practice linkage between public firm default probabilities and the Fed’s CCAR macro factors produces results that are both highly plausible and consistent with best practice econometrics. Consistent with the oft-stated bankers’ political objectives, the projected losses are worst in the severely adverse scenario and best in the base line scenario. The procedures used avoid the errors we often see in lowest common denominator credit modeling efforts. There is no need to pull any analytical punches in the CCAR process. Indeed, if ever there were a time when best practice is called for, risk assessment at the peak of the credit cycle is the best possible time to use best practice.
Angrist, Joshua D. and Jörn-Steffen Pischke, Mostly Harmless Econometrics, Princeton University Press, Princeton, New Jersey, 2009.
Stock, James H. and Mark W. Watson, Introduction to Econometrics, second edition, Pearson Education, Inc., Boston, Massachusetts, 2007.