• • Feedback from internal & external constituents
  • Back-testing, etc.

It’s especially important if the original CECL models were built on “older” firm data that stopped a couple years ago or incorporated non-firm, industry data

2. Renew—involves normal updates to

• • Regression coefficients
  • Historical averages

As new observations—say, the last four, six or eight quarters—are added to the historical sample set, a.k.a. the developmental database, it makes sense to update those estimates (and to see if they remain valid given the new observations).

3. Revise—when issues arise (usually with renewal)

• • Updated regressions may no longer pass relevant criteria or tests, or
  • Economic variable forecasts that were available during development, may no longer be available,; for example, if a supplier of economic data changes or eliminates an index that is used as a parameter in a regression equation.

4. Redo—when the initial implementation can’t be patched

• • Initial experience and internal or external feedback may indicate lack of representational faithfulness, reliability & conceptual soundness. (We’ve heard of places planning to redo their CECL models even before the initial implementation.)

5. Integrate/Extend—for advanced firms that realized:

• • Their extant risk ratings models and systems, as well as their pricing models, aren’t necessarily consistent with expected loss recognition (via CECL over the life of the loan). In fact, if the initial CECL development work was performed properly—meaning it is well-organized and documented—then it can be used to eliminate those inconsistencies. Development work (performed with enough foresight) should be as a long-term, reusable investment and not as a disposable, single-use expense.
  • That it’s possible to port CECL models to stress testing–although “port” doesn’t mean directly apply.

Given the benign economy through mid-February—and the expectations that those conditions would continue for a while—we anticipated that most CECL-related work would involve (1) reviewing and (2) renewing, with the occasional (3) revision. We also anticipated that at some time in the future, (i) a recession would illuminate model weaknesses that didn’t appear when losses were a few basis points per quarter, and (ii) consequently, bank executives would realize that their firm’s CECL models were less representationally faithful and reliable than first assessed, and, therefore, might call for redevelopment, especially if outside parties–auditors, the SEC or bank regulators–also noticed the weaknesses.

Now, however, with this incredibly quick Corona Virus shock, those weaknesses should now be visible. Of course, in the current panic—and with so many other, non-modeling issues also arising from this extreme situation—said weaknesses may not be immediately recognized.¹

By now, any interested party should know that the CARES Act allows banks to delay the implementation of CECL until the end of the declared national emergency or the end of the year, whichever comes first. We suspect that during the past few weeks, many bank executives considered whether to delay or not. While the novelty and extremity of the situation may increase the desire to delay, it’s hard to interpret a delay as anything less than an implicit admission of a non-robust CECL implementation (or, in some cases, just plain mistakes in modeling or processing, including scenario design, in the rush to present preliminary results to present for a delay decision). As we report in another post, to date—as of April 20—all the large banks are using CECL.

Of course, that doesn’t mean that application to the present environment is easy…

2. The Current Situation

With the recent end of the first quarter, many banks have or will make their final Q1 CECL run shortly and will have their ALLL/CECL Committee meetings, shortly, if they haven’t already.²

One of the key questions is: what will they run?

Those who are in a hurry will get scenarios from their providers—frequently external vendors—and use those unadjusted. However, there’s a problem with that. Our current predicament can be described in a single word: “unprecedented.”

1. An unprecedented modern, global pandemic;
2. An unprecedented quarantine of our economy;
3. An unprecedented governmental response–the CARES Act–to a brand new and shocking economic downturn; and
4. Unprecedented forecast scenarios.

During the “reasonable and supportable” forecast period, almost every bank is using a suite of statistical models; however, they weren’t built for the, unprecedentiveness, er, novelty, of the situation.

2.A. Scenario Design, Models and the Corona Virus

While most firms buy forecast scenarios—some implicitly through the purchase of models—some larger ones develop their own or combine both internal model output and vendor forecast scenarios.³ There are three important points to notice:

1. No extreme economic forecast regarding the Corona Virus can be built on models. Why? Because this is an unprecedented situation, which means that nothing like it is in the historical sample set used to develop those models. Any honest provider should explain that there is nothing “scientific” or empirical about their projections. The shocks might be well-formed, educated guesses, but they’re still guesses. That fact, alone, doesn’t mean that the scenarios should be discarded–just that they are more suspect than usual, and, therefore, subject to scrutiny and sensitivity analysis.⁴
2. Regardless of how the scenarios were generated, avoid extrapolation. That means don’t use forecasted parameters in your regression equations that are outside of the historical sample set. This may limit the severity of shocks, but (a) if you don’t, your CECL model output is complete and utter nonsense, and (b) quality adjustments—a.k.a. model overlays—can be made later to increase or decrease the severity of the expected loss.

Avoid extrapolation by processing scenarios prior to use (in your regression equations). For each parameter used in those equations, both the absolute levels within the scenarios and any transformations should be inspected and limited to the minimum or maximum found in the historical sample set. Anything else is simply invalid. Even if transformed parameters are within historical limits, if the absolute level has not been seen in history, this fact should be reported (to interested parties). This is a best practice that should be part of standard, quarterly procedures, regardless of environmental conditions.⁵

3. Many developers, validators and analysts are asking: “we’re seeing large shocks in weekly unemployment claims (or interest rates or forecasted GDP in scenarios or equity indices), but the CARES Act and both the federal government’s and the Federal Reserve’s efforts are supposed to mitigate the negative consequences of these shocks. What should we do? Other than making the adjustments you recommend, above, what else should we do?” Put another way, “how should we adjust our scenario when we know that some unemployed people will receive all (or even more than) than their normal compensation?”

Our advice to modelers is to adjust the scenario shocks as described in #2, so that the models remain mathematically—logically—valid and make adjustments through qualitative factors, which are also known as model overlays.

This is not only an unprecedented situation, but there is enormous uncertainty about its implications. These aren’t the same notions. A volcano could raise up and destroy a large, American city. That would be unprecedented, but there would be little remaining uncertainty; the city would simply be gone, and we would quickly know exactly what was lost.

Nothing can be done to eliminate the current uncertainty. Be patient. Make your best guess, today, and make revisions as time passes and more is learned. Run multiple scenarios and sensitivity analyses to inform those making quality adjustments.

While we would anticipate that economic forecast committees would make adjustments, based upon the committee’s collective view of the bank’s operating environment, any other adjustments—by, say, modelers—introduces control issues, and for something like CECL, that means SOX issues, too.

2.B. Qualitative Adjustments/Model Overlays

While no one could have anticipated the current situation, it is possible to have a robust, qualitative adjustment framework that could easily handle it. We know. We built such a framework, and it’s being used effectively and efficiently.

The advantage of well-designed, robust, qualitative adjustment framework—a.k.a. CECL model overlay framework—is that it eliminates any appearance of opportunism or expediency. For example, as described, while the shocks to, say, unemployment or GDP are unprecedented, so is the government’s response to mitigate the negative effects. Without a framework in place, adjustments to reduce losses may appear to be earnings manipulation or evidence of other control issues to outsiders. With a framework, necessary adjustments are just a part of normal operating procedures.

By the way, a robust, qualitative adjustment framework would look at recent trends—even if those observations are in the developmental database.

A state-of-the-art qualitative adjustment framework would incorporate results from the bank’s other, comprehensive stress-testing activities related to SR 12-7, if those exist, of course.

2.C. Are You Sure You Want to Use Those Economic Parameters?

We write elsewhere—here and here—why the unemployment rate is a poor predictor of commercial credit losses, especially for risk management.

We also suspect that many bankers and developers are regretting their choice of using an equity index to help forecast C&I losses; recall, again, how situations like this expose weaknesses. When an index, like the Dow-Jones Industrial Average (DJIA), represents well-considered and reasonable future expectations, it seems like an appropriate parameter. Alas, that’s not always the case; it’s a reasonable index, until it’s not, as in when it represents sheer panic: eek! eek!

So why do banks use the unemployment rate and equity indices, e.g., the DJIA, to forecast commercial loan losses? Our short answer: the Fed and CCAR.

The unemployment rate and the DJIA are two of the 16 domestic economic variables that the Fed forecasts; they’re not all independent, by the way; so, there’s actually a lot fewer than 16. If you read the Fed’s CCAR narratives, they seem to like unemployment, a lot, although we argue that they misread history, or at least, don’t weigh the long term properly. Regardless, CCAR came before CECL, and it’s (at least) reasonable to infer the the Fed encouraged CCAR model developers to incorporate unemployment—and, given the dearth of other variables, implicitly DJIA, too.⁶ CCAR model developers and validators often became CECL developers and validators. Indeed, if CCAR models weren’t directly ported to CECL, it’s still easy to believe those folks brought any previously developed biases with them…

While porting stress testing models to CECL, and vice versa, makes complete sense—when done intelligently—it should be entirely obvious that CECL isn’t CCAR. The problem with CCAR is that it is a completely hypothetical, regulatory exercise, and only the final—at the end of a long sequence of calculations or transformations—loss numbers are revealed publicly, and even then, they’re not used for much. While regulators may challenge these numbers (and the processes used to generate them), they receive little-to-no public challenge from investors or analysts. Those numbers aren’t just hypothetical, they’re academic in the worst sense of the word.

So, given (i) CCAR is completely hypothetical in nature and (ii) it’s a regulatory exercise and (iii) initially—at least during the first few years—the Fed seemed to emphasize showing large expected losses, it’s easy to see how it was possible for conservatism seep into the CCAR loss estimation. That doesn’t just involve reporting absolutely large losses, which was not discouraged, but it also involves choosing parameters that have maximum sensitivity (to parameter shocks), even if those parameters aren’t the most realistic or intuitive. (As we say, when thought is absent before calculation, you’ll get results, but they won’t necessarily make sense. There are plenty of regress’n fools out there.)

3. What’s Next?

So, where does that leave us? While many bankers thought there was little left to do with CECL, we anticipate their perspective will change.

1. Review has already intensified and will be further formalized, including broader and deeper reviews related to qualitative overlays.
2. With the recent shocks in economic parameters, Renewal will be sooner than otherwise planned.
3. Those shocks and changes in relationships, which may have seemed steady for the last decade or so, along with increased scrutiny by executives, boards and outsiders, will create the need for Revisions.
4. An increased number of banks will attempt to learn from their experiences and incorporate wholesale revisions—will Redo—their CECL implementations.

We’re here to help—either with informed advice or with our excellent team building a complete and foolproof CECL implementation, or more broadly, a loss-forecasting framework that incorporates both expected loss (CECL) and stress-testing.