Phase 1:
Model governance and compliance is often missed when financial institutions perform model validations.
Currently, with the new CECL standard requiring complex methodologies, support, and documentation during every step of the process, it is more critical than ever to perform the following functions as part of a model governance and compliance review:
- Policy review
- Evaluation of internal controls
- Model documentation
- Roles and responsibilities
- Ongoing monitoring
An appropriate governance program should include board and management oversight with updated policies and procedure and defined roles and responsibilities. The program should assess the performance of the model on an ongoing basis and should clearly state the model documentation and validation standards that are to be upheld.
Phase 2:
CECL models require clean, accurate model data inputs to ensure meaningful results. The approach to this phase should focus on the following areas:
- Review of loan data
- Historical data review
- Segmentation of loan pools (including credit quality characteristic analysis)
- In particular, it is critical to be confident in the completeness and accuracy of the historical data. Inputting data that is questionable or improperly vetted is ultimately a waste of time and the start of a futile process.
Phase 3:
Model assumptions and back testing stresses the importance of incorporating forward-looking data into the calculation of expected losses over the life of a loan. Forecasted economic conditions and voluntary and involuntary (default) prepayment rates are the main drivers of an institution’s model assumption set.
As for the approach to model assumptions, we suggest validating four primary areas:
- Economic forecast assumptions
- Default rates
- Prepayment speeds
- Discounting methodology
Qualitative factors are another key assumption that will impact an institution’s CECL estimate outside of the modeling. Typically, qualitative factors are included after the modeling is complete and the organization adjusts its historical estimate based on a variety of situations. Some of the major factors consist of changes in lending policies and procedures, changes in economic and business conditions, changes in past-due loans, and changes in the value of underlying collateral to name a few. Since these factors are typically applied outside of the CECL modeling, many validations overlook this area. However, these factors will have varying impacts to an institution’s final CECL estimate based on their region, organization structure, and historical loss experience.
Additionally, many institutions are bringing over their previous qualitative factor methodology and applying it to their CECL methodology. However, institution’s should be careful to exclude certain factors that may be embedded in the historical loss and forecasting calculations. For example, many institutions may apply an unemployment or housing value forecast to their life of loan calculation. Therefore, any adjustments to economic or business conditions applied after modeling as a qualitative factor may need to be removed from the process.
Phase 4:
Model methodology and testing is, in our perspective, the most crucial part of a valuable validation implementation. There are two types of validations – standard, which is a basic testing of instruments and possibly a few shadow calculations; and replication, which requires obtaining all CECL sets and assumptions used by the institution using those data sets and assumptions to independently model a CECL estimate. and then comparing the two results, category by category.
The basic approach to model methodology and testing entails:
- Review of model framework
- Full replication of model
- Category-level reviews
- Assumption testing
- Scenario and stress testing analyses
- Back-testing
The types of findings that are commonly produced through a replication approach to model validations g are user errors, process errors, data errors and methodology differences. By replicating the model, we can uncover these errors and help resolve the issues to enhance the model for future iterations.
Additionally, the model validation offers a variety of replication benefits and value, such as giving the client full assurance on “black box” calculations and allowing the client to learn more about how assumptions and methodology impact their profile. From an internal perspective, the approach allows an institution to validate all parts of the process from start to finish. Not to mention, it helps ensure more data quality control, as noted earlier.
Phase 5:
Once an institution’s model process is set, we highly recommend performing stress tests and scenario analyses prior to implementation. These steps help gauge the impacts of movement in assumptions, such as increases and decreases in prepayment speeds, higher and lower severities, and increases and decreases in discount rates.
When running scenarios, we believe it is best practice to include a worst-case scenario (or “break the bank”) . Understanding the scenarios and the forecasting metrics will help an institution prepare not only for all real-life scenarios, but also for all types of questions from regulators, auditors, and other interested parties. Plus, the data gained from these scenario analyses can aid with capital and budgeting discussions down the road.
For a more detailed explanation, watch our informative webinar
CECL impacts organizations in a variety of ways. Connect with our team of CECL specialists to help your organization with its CECL needs.
