Have you ever tried to explain how an algorithm works to someone? It’s tough. It often feels like a mysterious black box. You put data in one end, and a decision—like a premium price or a claim approval—comes out the other.
For years, that's been a growing challenge in our industry. We all know insurance companies are leaning heavily on artificial intelligence, and honestly, for good reason. It can make things faster and more efficient. But for regulators, whose job is to protect consumers, that black box is a real problem. How can you be sure an AI system is fair if you can't see how it's making decisions?
Well, it looks like regulators are about to get a new pair of X-ray glasses.
The National Association of Insurance Commissioners (NAIC) is rolling out something new, and it’s a big deal. Let’s talk about what’s happening.
What is This New AI Tool, Anyway?
Think of it less like a rigid rulebook and more like a new diagnostic tool for a mechanic.
A special group within the NAIC, the Big Data and Artificial Intelligence Working Group, has developed what they’re calling the "AI Systems Evaluation Tool." The idea was floated over the summer, and now it’s moving into the real world.
Iowa Insurance Commissioner Doug Ommen, who’s been a key voice in this, put it pretty simply. He said regulators aren't being forced to use it, but it's another option they'll have when they perform market conduct exams.
“We just view that this tool is all part of that process to better understand what it is that companies are doing and what controls are in place,” Ommen explained during a recent meeting.
Basically, when regulators come in to check on an insurer's practices, they can now use this tool to ask smarter, more targeted questions about the company's AI. It's designed to help them peek under the hood and understand the logic, governance, and risks associated with these complex systems.
They're Kicking the Tires First with a Pilot Program
This isn't some sweeping mandate that's being dropped on the industry overnight. Regulators are being smart about this. They’re starting with a pilot program to see how the tool actually works in practice.
Ommen was clear that they don’t want to be "too rigid" on how it's used right now. The goal is to learn.
So, for the pilot, different states will use the tool during a mix of financial and market conduct exams. Afterward, they’ll all get together and share notes. What worked? What was confusing? Where could the questions be clearer? It’s a classic feedback loop, which is exactly what you want to see with something this complex.
To get everyone on the same page, the working group is even holding a special four-hour session to go through the tool line by line. That's happening on Sunday, December 7th, at the NAIC 2025 Fall Meeting down in Hollywood, Florida.
Why Is This Happening Now?
If you’ve been in the insurance world for more than a few years, you know this has been coming. The use of AI isn't new, but the level of sophistication has exploded.
A decade ago, we were talking about basic automation. Now, we’re dealing with generative AI and advanced models that are woven into everything from underwriting and pricing to customer service chatbots and claims processing.
Through all this rapid change, regulators have been trying to figure out how to build guardrails. Their core mission hasn't changed: ensure fairness, transparency, and consumer protection. But applying those old principles to this new technology is a whole new ballgame.
This new tool isn't their first swing at the plate. You might remember that back in December 2023, the NAIC adopted a "Model Bulletin" on the use of AI. But that was more like high-level guidance. It was a document meant to remind insurers to use AI in ways that align with existing laws on fair trade and corporate governance. It was a good start, but it wasn't a practical, hands-on tool. This new evaluation tool is the logical next step.
So, What's Different About This Tool?
A couple of law firms that keep a close eye on these things, Locke Lord and Eversheds Sutherland, have already pointed out a few key things that make this tool significant. Let me break it down for you.
1. It’s a Wider Lens
The 2023 bulletin was mostly focused on preventing bad outcomes for consumers, which is obviously critical. But this new tool casts a much wider net. It looks at a whole range of risks, including financial and financial reporting risks that could come from AI systems. It’s a more holistic checkup on the insurer’s overall health, not just one specific symptom.
2. It's Built to be Flexible
This isn't a one-size-fits-all questionnaire. The tool includes templates, charts, and checklists that regulators can adapt. This is so important because Company A might use a simple algorithm for marketing, while Company B has a deeply complex AI running its entire underwriting department. A rigid form would be useless. This flexibility allows regulators to customize their review to fit the situation.
3. The Goal is Understanding, Not Just Punishment
This is probably the most important point. The tool is intended to be a resource to help regulators learn. It’s about increasing their understanding of how different companies are using AI and managing the risks. It’s a starting point for a more informed conversation, not just a "gotcha" checklist to hand out fines.
So, what does this all mean for us?
It means that the age of AI as an untouchable "black box" in insurance is coming to an end. Regulators are getting more sophisticated, and they're building the resources they need to ask the tough questions.
For insurers, this is a clear signal to double-down on AI governance, documentation, and risk management. If you can't explain how your models work and what you're doing to ensure they're fair and accurate, you're going to have a tough time in your next market conduct exam.
Honestly, I think it’s a positive step for the industry. More transparency and accountability will ultimately build more trust with consumers, and that’s something we can all get behind. We’ll definitely be keeping a close eye on how this pilot program unfolds.
