Building Confidence in ALLL Models – a Timely Practice

Building Confidence in ALLL Models – a Timely Practice

Issue 

As illustrated during the global credit crisis, setting reserves with allowance for loan and lease loss (ALLL) models can be highly difficult. From 2000-09, ALLL reserves did not grow commensurately with risk: Commercial bank reserves grew by 2.3 times, while the charge-off volume of these loans and leases grew by more than five times.1 Financial companies are viewing these developments as an opportunity to revisit and build confidence in their ALLL models.

A model redevelopment or review process allows management to gain transparency into and become more comfortable with an ALLL model’s quantitative mechanics. Firmer comprehension and trust in the model allow management to focus its expert judgment where it is relevant and most needed – on qualitative adjustments. By understanding the components and inherent limitations, management’s evaluations are better informed.

Challenges  and Opportunities

Uncertain economic events can serve as a painful yet valuable reminder of the importance of ensuring that ALLL models reflect current conditions. Periodic recalibration removes or adjusts established model assumptions that may be latent or even harmful. Financial companies should review for relevance inputs such as historical data or legacy tools that result from acquisitions. By challenging each model element and confirming or refuting its necessity, these organizations can ensure the resultant product contains well-understood and transparent assumptions.

Forecasting the future is not an easy task; forecasting future loss dynamics poses additional challenges. The following components are key obstacles that require periodic calibration:

  • Portfolio Composition – New product offerings or renewed emphasis of an existing one can alter loan type distribution in a portfolio. Models with portfolio assumptions require an adjustment parameter to reflect dynamic portfolio mixtures. As an example, over the course of the past decade, subprime loans began to make up a larger percentage of financial companies’ portfolios. Any economic decline has a greater impact on these types of loans than it does on others. Hence, an economic adjustment factor is needed to account for this information. In other cases, the purchase of new portfolios originated by another lender introduces new underwriting standards into the organization’s portfolio, requiring further adjustment.
  • Operational Strategy – Modification of existing loan collecting procedures will influence loan delinquency behavior and may render the use of historical data erroneous. For example, a shift to call customers 15 days after first delinquency date instead of 45 days will result in this segment of loans behaving in a markedly different manner compared to historical performance. Model adjustments to reflect this effect include the use of proxy data, regression techniques on existing data or extrapolation with similar segments.
  • Economic Environment – Most ALLL models are based on historic performance data and are backward-looking. Financial companies should incorporate forward-looking data, such as unemployment rates, interest rates or the consumer confidence index, to allow for a quantitative framework that describes qualitative trends.
  • Data Limitations – Building forecast models without abundant data requires creativity and intuition – notions that can be dangerous when building an objective ALLL model. This is especially true when a financial organization utilizes subtle statistical techniques, which inevitably bend the shape of the credit portfolio towards the conditions at the time of development. It is often the case that these assumptions remain embedded deep within the model. Model review revisits the interpretation to assess its relevance.

Our Point of View

Across the financial services industry, more organizations are recognizing the value of ALLL model evaluation. In addition, these models recently have been under heavy regulatory scrutiny. Regardless of the motivation, organizations can benefit from a review because it allows them to gain confidence in their ALLL models. By taking a fresh, objective look at these models and ensuring they remain relevant for current conditions, a financial company will gain insight and transparency into its model’s quantitative framework, as well as an understanding of the model’s assumptions and limitations.

How We Help Companies Succeed

Protiviti’s Model Risk Management practice helps organizations understand and improve their ALLL models. Experts in our practice leverage their industry experience and heavy quantitative background to utilize an approach that meets our clients’ needs. This approach also is based on regulator expectations, including regulatory guidance such as OCC 2000-16. We offer three types of services:

  1. Model Development – After understanding the client’s needs, we build a model that fits into the existing operational infrastructure. In some cases, clients prefer a champion/challenger format, where the newly developed model is complementary to an existing one. Models we have worked with include vintage-based, roll rate, and Markov chain models, as well as PD/LGD frameworks. Often times, redevelopment is not practical, so we augment a current model by including forward-looking inputs so that a forecast is not reliant completely on historic trends.
  2. Model Validation – Evaluation of a model can uncover antiquated data interpretations. Especially in older models, we often discover layers of adjustments that are no longer relevant and are embedded in the output management utilizes. We seek to remove irrelevant assumptions so that management can make informed decisions. The Model Validation process also includes a Model Verification.
  3. Model Verification – Management may be comfortable with the assumptions and theoretical platform, but may not have confidence in the implementation process. Implementation errors range from inadvertent calculation mistakes to misuse of data inputs and reporting misinterpretations. Through verification, we focus on understanding the ALLL model through interviews with stakeholders, review of documentation and model code, and creation of an independent model for comparison of outputs.

Example

Management of a retail lender was concerned with the size of its expert judgment adjustments to its reserve. It sought assistance in evaluation of the quantitative component of its ALLL model. Through documentation review, interviews and code review, our team independently created a proxy model. We compared the outputs of the two models and exposed several inconsistencies. Our investigation uncovered multiple layers of model code along with a patchwork of “quick” fixes used to conform to business events. Over time, the organization had accepted these layers as part of the model, which was not being reviewed periodically. Our holistic evaluation allowed our client to ensure its model remained relevant and thus build confidence in its assumptions and outputs.

Note: FDIC website link.

Contacts

Shaheen Dil, Ph.D.
+1.212.603.8378
[email protected]
 
 
CATEGORY TOPIC:
CATEGORY INDUSTRY:

Ready to work with us?