When we talk about a best practice default probability model, we mean the same thing whether we are talking about a retail default probability model, a small business default model, a sovereign model, or a corporate default probability model. For background on state of the art default probability modeling, please see the Kamakura Risk Information Services sovereign and corporate default model brochures on www.kamakuraco.com. The characteristics of a best practice default model can be summarized like this:
- The model is based on a large sample (see our blog post from April 14, 2009 on “Building a default model: lessons learned about how much data is necessary”)
- The time interval used for modeling is monthly (best) or quarterly (second best)
- The data set for modeling spans a long time period, at least one full business cycle
- The fitted model includes macro-economic factors in the underlying set of explanatory variables and in a “reduced reduced form model” used for simulating forward over a large set of counterparties (see our blog post from March 14 on “FAS 157 Valuation and Macro Factor Stress Testing: Why "Reduced Reduced Form" Default Models are Essential”)
- Default probabilities are updated daily based on most current inputs
- Default probabilities are produced to the nearest basis point
- Default probabilities have a full term structure, from 1 month to five years or more
- Accuracy tests, coefficients, and inputs are revealed to clients and regulators in a fully transparent way
How does such a best practice default model compare to agency ratings (or internal ratings) or a credit score like a FICO score, a commonly used consumer credit score in the United States offered since 1956 by FICO (for more information see www.myfico.com)? We ask and answer a number of questions that outline the distinction between the measures:
What is the maturity of the credit measure?
Default probability: explicit maturity is given
Credit rating: ambiguous
Credit score: ambiguous
What default probability is associated with the credit measure?
Default probability: explicitly given
Credit rating: ambiguous
Credit score: ambiguous
Is the credit measure an ordinal or absolute measure of credit risk?
Default probability: both
Credit rating: ordinal
Credit score: ordinal
Does the credit measure have a term structure that measures risk over many time horizons?
Default probability: yes
Credit rating: no, since the time horizon is ambiguous and only one value is given
Credit score: no, since the time horizon is ambiguous and only one value is given
Does the credit measure have explicit macro factor inputs like those recently mandated for stress testing by the U.S. bank regulators (real GDP, unemployment, home prices)?
Default probability: yes
Credit rating: no
Credit score: no
Is the accuracy of the credit measure public information?
Default probability: yes (see The Basel Handbook, second edition, 2007, RISK publications)
Credit rating: yes for bonds, no for collateralized debt obligations
Credit score: no for the general public
How often is the credit measure updated?
Default probability: daily
Credit rating: every 6-18 months on average
Credit score: approximately monthly according to www.myfico.com
How many gradations are there in the risk measure?
Default probability: 10,000 (from 0.00% to 100.00% in basis points)
Credit rating: 20
Credit score: 351 (from 300 to 850 in 1 point increments according to www.myfico.com)
Can the risk measure be independently calculated by a third party auditor, regulator, or risk model audit department?
Default probability: yes
Credit rating: no
Credit score: no
Is the process by which the credit measure is determined fully automated or does it involve human judgment, committee meetings, and potential errors or variations in judgment?
Default probability: fully automated
Credit rating: human judgment
Credit score: fully automated
How often are the weightings of the inputs to the model updated?
Default probability: on demand, as needed. Generally annually
Credit rating: ambiguous
Credit score: every few years
Is there a broad base of academic theory that links the credit measure to valuation and hedging of a particular extension of credit?
Default probability: yes
Credit rating: no
Credit score: no
Can one link the credit spread on an instrument to the credit measure?
Default probability: yes (see Jarrow et al, RISK, September 2007)
Credit rating: ambiguous
Credit score: no
This long series of questions and answers makes it clear that ratings and credit scores are inferior to a best practice default model in nearly every respect. They are to a best practice default model what a hand-held calculator is to a modern laptop computer. They are accidents of history that are, for the most part, unnecessary if one has a modern best practice default model. Our recent blog post (“A Ratings Neutral Investment Policy,” May 12, 2009) explains how an institution can set investment strategy without reference to traditional ratings, for example.
Credit scores and ratings have meaning only in a fairly limited set of contexts:
- Lenders who have no experience with default probabilities have at least some familiarity with credit scores and ratings, so they have value as stepping stones to best practice credit risk assessment
- Ratings (but not credit scores) can incorporate “one off” changes in risk that are so rare that they would not normally be incorporated in a modeling data base for default probabilities or credit scores. Examples would be Steve Jobs’ health at Apple and fraud at Enron.
- Credit scores are “politically correct” in that they don’t upset retail borrowers as much as the statement “Your probability of default is 10%.” By obscuring the accuracy of the credit measure, one can discuss it with borrowers. That being said, as www.myfico.com explains, almost every bank overlays other factors on top of the credit score in its final credit assessment
Credit scores and ratings can be derived from default probabilities quite easily (see the KRIS Technical Guide, Version 4.1 and the Kamakura Implied Ratings Brochure on www.kamakuraco.com for examples) but the reverse is much more complex and much less accurate than deriving default probabilities directly without using ratings or credit scores as the primary input to a default model. From the perspective of 2009, both techniques are relics of the 20th century, and we’ve moved on to better technology.
Donald R. van Deventer
Kamakura Corporation
June 16, 2009