Understanding the supervisory guidance on model risk management for anti-money-laundering systems

Although federal regulators have provided some guidance around model validations for anti-money-laundering (AML) systems, some financial institutions may not understand what to expect when their systems are validated.

Background

On June 7, 2017, the Federal Deposit Insurance Corporation (FDIC) adopted the supervisory guidance on model risk management originally issued by the Office of the comptroller of the currency (OCC) and the Federal reserve.

With the FDIC’s adoption, three major bank regulatory bodies have adopted guidance pertaining to model validations. This means institutions could start receiving guidance or scrutiny during Bank Secrecy Act (BSA) examinations.

While the original bulletin focused on credit, liquidity, and capital applications, it’s being applied to AML systems as well to help make sure institutions:

• Adequately assess risk
• Properly use the AML system

Frequency of model validation

The guidance states that financial institutions should complete a model validation at least annually. While this timeframe could be extreme in some cases, it’s important to note that AML model validations do need to be completed on a periodic basis.

Model validations aren’t something that’s completed once and then the regulatory requirement is met. The frequency at which model validations need to be completed is based on the complexity of the institution and if there’ve been any recent critical changes effecting the system.

Examples of critical changes that may prompt an AML-system validation include the following:

• Upgrade or change to core systems
• Change in the BSA or AML risk profile
• Changes to the customer base
• Addition of a new product or service
• Completion of a merger or acquisition
• Major changes to the AML system
• Guidance from a primary regulator
• Findings from a prior model validation

Review areas

Whether an independent firm or in-house staff is completing the model validation, the following four areas should be reviewed.

Data integrity

This is arguably the most important step of any model validation. The results of analysis won’t have any meaning behind them without knowing the data being analyzed is of high quality and complete and accurate.

The data-integrity process includes two main steps:

• Obtaining data from all the systems that feed into the AML system
• Verifying data that’s been modified to be processed by the AML system still accurately reflects original information

Data validation

After verifying the integrity of data to be used in the analysis, institutions should test the alerts the AML system generates by doing the following:

• Review the alerts the system produces (system alerts)
• Recreate new alerts to compare to the system alerts (test alerts)

There will often be discrepancies — test alerts generated that the system doesn’t produce or system alerts that weren’t able to be recreated. These results require additional analysis around why an alert did or didn’t happen.

Tuning or reasonableness

The third step — and arguably the most difficult — is to determine if current thresholds are adequately reflective of the institution’s BSA or AML risk profile and to tune them, if needed. This step is very time intensive and requires extensive documentation.

If the thresholds are changed, and especially if they’re lowered, there has to be sound, documented justification for why as well as a level of certainty that any major money-laundering scheme will still be detected.

Governance

It’s impossible to have a perfect model because of one or both of the following reasons:

• There will always have to be assumptions input by users
• There’s a system limitation preventing it from running a perfect model

That’s why it’s important to understand and address the assumptions and limitations of an AML system.

The Office of the Comptroller of the Currency (OCC) noted within their bulletin that the following areas are required for proper model governance:

• Board and senior management
• Policies and procedures
• Roles and responsibilities
• Internal audit
• External resources
• Model inventory
• Documentation

By knowing and understanding what limitations exist, an institution can implement a proper governance structure to discuss how to further mitigate risks and respond to events that could adversely affect the organization.

Next steps

Institutions can complete this process internally or outsource to a firm. However, if choosing to perform the analysis in-house, the review should be performed independent from the normal management of the process.

When performing the analysis in-house, the following conditions should be met by the employees performing the validation:

• They don’t use the system
• They don’t tune or modify thresholds
• They weren’t involved in selecting the system

As the regulatory burden continues to increase, institutions will be expected to have their AML systems validated regularly and be able to understand how to address identified issues.

Whether this is completed in-house or outsourced to a firm, the same four major components must be addressed and any gaps within the system mitigated based on an organization’s risk profile.