Blog

The Takeaway

A bill has been introduced in the Illinois legislature that would mandate the use of human oversight on a health insurer’s negative coverage decisions. It’s likely that other types of insurers will soon encounter similar oversight attempts.

Overview of the Bill’s Purpose

Illinois lawmakers are again at the forefront of regulating emerging technology. While it seems inevitable that utilizing artificial intelligence will become part of decision-making processes, Illinois State Representative Bob Morgan is leading the effort to require that human decision-makers oversee insurance coverage decisions made by AI systems. In January, Morgan introduced House Bill 35 (HB 35) – the Artificial Intelligence Systems Use in Health Insurance Act. This Bill could serve as a roadmap for how Illinois regulates the use of AI in healthcare and other areas where the human factor is vital to the outcome. If passed, the Bill could have lasting implications for all insurers, not just those providing personal healthcare insurance.

Synopsis of the Bill’s Provisions

HB 35 would bar insurers from issuing an adverse outcome on an insured’s health insurance claim based solely on AI or a predictive model. Those adverse outcomes are defined as specifically including those “with regard to the denial, reduction, or termination of insurance plans or benefits.” The Bill would require that an individual with authority to override any AI system’s determinations conduct a “meaningfully review” of any adverse decisions. Furthermore, that review should be conducted in accordance with rules which the Department of Insurance issues.

The Bill empowers the Department to make rules to implement the Bill’s provisions. It also allows the Department to provide additional guidance (e.g., criteria or a framework that better defines “meaningful review”). Likely, the health insurance industry will be looking for opportunities—through rule making and other less formal means—to provide input on what constitutes “meaningful review” as that standard is nebulous and could be applied subjectively.

The Bill also empowers the Department to review insurers’ use of AI in the development and implementation of AI systems in their businesses in Illinois. A “review” could involve a Department investigation or “market conduct action regarding the development, implementation, and use of AI systems or predictive models” and the outcomes from their application. Additionally, acknowledging the speed at which insurers are developing and implementing AI, the legislation authorizes the Department to adopt emergency rules to implement the Act.[1] Acknowledging the need for transparency, the Bill would also allow the Department to adopt rules for full and fair disclosure of an insurer’s use of AI.

Potential Implications if the Bill Passes

This proposed bill signals a shift towards cementing into law protections like those announced in the NAIC model bulletin. HB 35’s provisions highlight the potential adverse consequences of removing human judgment from decisions concerning health insurance coverage. It acknowledges the need to balance the benefits of an insurer’s use of AI with the potential for lack of human control over adverse coverage decisions for insureds.

While the Illinois legislature is first turning its attention to oversight of healthcare insurers, we expect other insurers will encounter similar oversight, particularly when coverage decisions impact individuals’ well-being. We also anticipate a significant push to ensure that human beings have the final word in insurance decision-making.

[1] In March 2024, Illinois adopted the National Association of Insurance Commissioners (NAIC) model bulletin on the use of AI.