Various states push momentum towards regulating insurer's use of artificial intelligence

On Sept. 21, 2023, the Division of Insurance at Colorado’s Department of Regulatory Agencies (Division), adopted Regulation 10-1-1, Governance and Risk Management Framework Requirements for Life Insurers’ Use of External Consumer Data and Information Sources, Algorithms, and Predictive Models. The regulation establishes requirements for life insurers that utilize External Consumer Data and Information Sources (ECDIS), as well as algorithms and predictive models that incorporate ECDIS.

Effective Nov. 14, 2023, the regulation, which applies to all life insurers authorized to operate in Colorado, mandates the establishment of a risk-based governance and risk management framework. The framework is designed to ensure that the use of ECDIS, algorithms and predictive models does not result or potentially result in unfair discrimination, particularly with respect to race, and provide a method to remediate unfair discrimination. Click here for more information on SB21-169.

The governance and risk management framework outlined in the regulation mandates life insurers using ECDIS, algorithms and predictive models to:

1. Formulate governing principles: Document guiding principles to prevent unfair discrimination in the use of ECDIS, algorithms and predictive models.
2. Ensure oversight: The board of directors or a suitable committee should oversee the risk management framework, with senior management responsible for setting strategy and monitoring performance.
3. Create a cross-functional team: Establish a team from key functional areas to oversee the use of these tools.
4. Develop policies and training: Create written policies for the design, use and monitoring of these tools, including a training program for relevant personnel.
5. Handle consumer complaints: Implement processes to address consumer complaints and inquiries about the use of these tools.
6. Assess risks: Develop a system for assessing and prioritizing risks associated with these tools, considering consumer impacts.
7. Maintain an inventory: Keep an updated inventory of all used ECDIS, algorithms and predictive models, documenting any changes.
8. Conduct testing and monitoring: Document testing conducted to detect unfair discrimination and monitor the performance of algorithms and predictive models.
9. Manage external resources: Document the process used for selecting third-party vendors supplying ECDIS, algorithms and/or predictive models, and ensure compliance with all regulatory requirements.
10. Review regularly: Conduct regular reviews of the governance structure and risk management framework, updating documentation as needed.

The updated draft regulation for life insurers has been paired down from the initial version released in February 2023. Most notably, the updated draft no longer emphasizes "disproportionately negative outcomes," which would have included results or effects that "adversely impact a group" with protected characteristics "even after considering factors that define similarly situated consumers." Instead of this term, the updated draft pivots to necessitating "risk-based" governance and management frameworks. This shift is substantial – it not only brings the updated draft in line with conventional insurance regulation, but also signifies a pragmatic, progressive advancement for such regulation.

However, despite being less demanding than the initial draft, the updated version still imposes significant obligations on life insurers. These include mandates for life insurers to set up risk-based frameworks for the utilization of ECDIS in any insurance practice including claims, ratemaking and pricing. Furthermore, the regulation necessitates the execution of these frameworks concerning any algorithms and predictive models that use or depend on ECDIS.

The regulation outlines comprehensive reporting requirements. Insurers using ECDIS, algorithms and predictive models must submit a narrative report to the Division, summarizing their progress towards compliance with the regulation's requirements. Conversely, insurers that do not use ECDIS or algorithms and predictive models are exempt from these requirements but must submit an attestation to that effect.

The regulation stipulates that sanctions may be imposed, including civil penalties, cease and desist orders, and/or suspensions or revocations of license, subject to the requirements of due process.

1. Understand the regulation: Familiarize yourself with the definitions and requirements of the regulation. Understand what constitutes ECDIS, algorithms and predictive models, and how they are used in your organization
2. Establish a governance and risk management framework: Develop a comprehensive framework that includes principles, responsibilities, a cross-functional committee, roles and responsibilities, policies and processes, training, controls, protocols for consumer complaints, a plan for unintended consequences and the use of external audits
3. Maintain comprehensive documentation: Keep detailed records of all ECDIS, algorithms and predictive models in use, including those supplied by third parties. This should include an inventory, results of annual reviews, a system for tracking changes, descriptions of testing, limitations, ongoing monitoring, datasets used, how predictions are made, potential risks and impacts, the process for selecting external resources and all decisions made regarding the use of ECDIS and algorithms
4. Prepare for reporting requirements: Plan for the submission of reports to the Division summarizing your progress towards compliance with the requirements specified in the regulation. These reports are due six months following the effective date of the regulation and annually thereafter
5. Plan for potential noncompliance: Understand the potential consequences of noncompliance, including civil penalties, cease and desist orders, and/or suspensions or revocations of license. Ensure that your organization has a plan in place to address any potential noncompliance issues

On Jan. 17, 2024, the New York Department of Financial Services (NYDFS) issued a proposed insurance circular letter to all insurers authorized to write insurance in New York state titled Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing. As noted by the Department, “the purpose of this circular letter (“Circular Letter”) is to identify DFS’s expectations that all insurers authorized to write insurance in New York State, licensed fraternal benefit societies, and the New York State Insurance Fund (collectively, “insurers”) develop and manage their use of ECDIS, artificial intelligence systems (AIS), and other predictive models in underwriting and pricing insurance policies and annuity contracts.”  

The letter goes on to state that the Department may examine the use of ECDIS and AIS within the scope of examinations. Per the Department, the use of ECDIS and AIS should be governed by Fairness Principles, such that “an insurer should not use ECDIS or AIS for underwriting or pricing purposes unless the insurer can establish that the data source or model, as applicable, does not use and is not based in any way on any class protected pursuant to Insurance Law Article 26. Moreover, an insurer should not use ECDIS or AIS for underwriting or pricing purposes if such use would result in or permit any unfair discrimination or otherwise violate the Insurance Law or any regulations promulgated thereunder.”  

The Fairness Principles outlined by the Department explicitly outline the following:  
Data actuarial validity: Insurers should be able to demonstrate that the ECDIS are supported by generally accepted actuarial standards of practice and are based on actual or reasonably anticipated experience   

Unfair and unlawful discrimination: Specifically noting the insurer should not use ECDIS or AIS in underwriting or pricing unless the insurer has determined that the ECDIS or AIS does not collect or use criteria that would constitute unfair or unlawful discrimination or an unfair trade practice   

Analyzing for unfair or unlawful discrimination: Insurers should perform and document regular testing of ECDIS and AIS for potential discrimination using both qualitative and quantitative considerations  

• Qualitative: The ability to articulate the ECDIS used, the nature and intent of the AIS and the potential disparate impacts which could result from its use  
• Quantitative: The use of specific metrics to monitor outcomes from the use of ECDIS or AIS, including (but not limited to):  

Adverse impact ratio: Analyzing the rates of favorable outcomes between protected classes and control groups to identify any potential disparities 

Denials odds ratios: Computing the odds of adverse decisions for protected classes compared to control groups 

Marginal effects: Assessing the effect of a marginal change in a predictive variable on the likelihood of unfavorable outcomes, particularly for members of protected classes 

Standardized mean differences: Measuring the difference in average outcomes between protected classes and control groups 

Z-tests and t-tests: Conducting statistical tests to ascertain whether differences in outcomes between protected classes and control groups are statistically significant  

Drivers of disparity: Identifying variables in AIS that cause differences in outcomes for protected classes relative to control groups. These drivers can be quantitatively computed or estimated using various methods, such as sensitivity analysis, Shapley values, regression coefficients or other suitable explanatory techniques  

In addition to the principles outlined above, there are also expectations around areas like governance and risk management, including:

• Board and senior management oversight
• Policies and procedures
• Risk management and internal controls
• Linkage to other risk management activities (e.g., ERM)
• Standards for model development, implementation, use, validation, review and testing
• Roles, responsibilities and competencies

Engagement of internal audit, which may include:

“-verifying that acceptable policies and procedures are in place and are appropriately adhered to;

-verifying records of AIS use and validation to test whether validations are performed in a timely manner and AIS models are subject to controls that appropriately account for any weaknesses in validation activities;

-assessing the accuracy and completeness of AIS documentation and adherence to documentation standards, including risk reporting;

-evaluating the processes for establishing and monitoring internal controls, such as limits on AIS usage;

-assessing supporting operational systems and evaluating the accuracy, reliability, and integrity of ECDIS and other data used by AIS;

-assessing potential biases in the ECDIS or other data that may result in unfair or unlawful discrimination against insureds or potential insureds; and

-assessing whether there is sufficient reporting to the board or other governing body and senior management to evaluate whether management is operating within the insurer’s risk appetite and limits for model risk.”

Adherence to all applicable laws, rules and regulations

Establishment of written standards, policies and procedures for the planning, vetting, acquisition, use and monitoring of EDCIS and/or AIS by third parties

Lastly, the Department outlines the transparency requirements of existing insurance laws and circular letters (Insurance Circular Letter No 1 [2019]) outlining among other things the specific requirements pertaining to refusal, limitation or differing rates stemming from certain conditions. The letter also outlines the requirements around notification for the reason behind declination, limitation, rate differential or other “adverse” underwriting decisions, as well as the potential insured’s right to request information used by AIS in the underwriting process.

The comment period for this letter closed on March 17, 2024.