A new report from the EU’s cybersecurity agency, ENISA, just dropped a bombshell: less than 15% of AI systems running in our critical infrastructure meet basic benchmarks for AI interpretability. That number shows just how big the gap is between theory and practice, and it throws the real-world regulatory challenges of building explainable AI into sharp relief.
Key Takeaways
- Just 15% of AI in critical infrastructure hits interpretability targets, a massive compliance gap.
- The EU AI Act is forcing design changes, requiring high-risk AI to have real interpretability built in from the start.
- Without standard metrics for interpretability, regulators can’t enforce rules consistently and developers are flying blind.
- Companies need to build teams with developers, lawyers, and actual subject matter experts who can talk to each other.
- Using regulatory sandboxes is a smart, hands-on way to figure out compliance and show regulators you’re on the right track.
Only 15% of Critical Infrastructure AI Systems Meet Interpretability Benchmarks
That low compliance rate, pulled from ENISA’s 2025 AI in Critical Infrastructure Security Report (ENISA, 2025), points to a systemic failure. We’re leaning more and more on AI to run everything from energy grids to transit networks for efficiency and security. But when these systems are “black boxes,” their decisions are a total mystery. Think about an AI managing a power grid that suddenly reroutes electricity with no explanation. In a situation like that, opacity can cause catastrophic failures and makes it nearly impossible to audit what went wrong or how to respond. The report calls out specific weak spots in aviation control and medical diagnostics, where a human absolutely must have clarity on why the AI made a certain call. The tech can do it. The problem is a deep regulatory void where the “how” of AI decisions gets completely ignored in deployment protocols. It’s clear that organizations just don’t know how to turn vague interpretability principles into hard, measurable features in their systems.
The EU AI Act Mandates Strong Interpretability Features for High-Risk Systems
The EU’s AI Act, which started its phased rollout in early 2026, is a huge legislative swing at these problems. Article 13 of the Act (Official Journal of the European Union, 2026) is explicit: high-risk AI systems must be built so their operations are transparent enough for a user to actually interpret the output. This means giving clear explanations for a model’s predictions, what factors drove them, and how the system behaves overall. For developers, this means they can’t just chase accuracy scores anymore. They have to build interpretability methods like LIME (Local Interpretable Model-agnostic Explanations) or SHAP (SHapley Additive exPlanations) into the development pipeline from day one, not tack them on as an afterthought. The real difficulty is in the term “sufficiently transparent,” which is dangerously subjective and means one regulator might approve what another rejects. This ambiguity forces developers to guess at how the law will be interpreted, turning compliance into a minefield. The act’s focus on human oversight is also critical, because that oversight is completely hollow without an AI you can actually understand. How can a person effectively supervise a system they can’t get answers from? They can’t. The EU AI Act is legislating that reality.
Lack of Standardized Metrics for Interpretability Hinders Enforcement
One of the biggest blockers to regulating AI interpretability effectively is that we have no standardized, accepted metrics. A 2025 NIST survey found that over 70% of AI practitioners see this lack of clear evaluation standards as the single greatest obstacle to building and proving interpretability (NIST, 2025). Right now, developers are stuck using qualitative gut-checks or ad-hoc metrics to decide if a model is “interpretable.” This has led to a fragmented mess where different companies, and even teams inside the same company, are all using different yardsticks. Regulators are then left struggling to apply the rules consistently. You can’t compare systems, audit for compliance, or share best practices if you don’t even have a common language to measure what you’re talking about. We need hard, quantitative measures for explainability, something like the accuracy or precision metrics we use for performance. The current dependence on subjective expert judgment is fine for one-off projects, but it doesn’t scale for broad regulation. People say interpretability is inherently qualitative, but I think that’s a cop-out. With focused research, we can absolutely develop quantitative proxies that allow for objective evaluation and consistent rules.
Interdisciplinary Teams Are Essential for Bridging the Communication Gap
The sheer complexity of AI interpretability means you need people from different fields working together. A study from The Alan Turing Institute (The Alan Turing Institute, 2026) found something telling: projects with teams of AI engineers, lawyers, ethicists, and domain experts had a 40% higher success rate in hitting their interpretability goals than tech-only teams. The communication gap is usually huge. The AI engineer is obsessed with model architecture, while the lawyer is worried about legal liability and GDPR. The domain specialist, say, a doctor using an AI diagnostic tool, needs an explanation that makes sense in the context of their clinical workflow. If you don’t have dedicated roles and a way for these people to talk, their different points of view never connect, and you end up with an AI that’s technically impressive but legally exposed and practically useless. Companies have to invest in training that gets everyone on the same page, teaching the legal team basic AI and the engineers the real-world regulatory consequences of their code. It’s about creating a shared language for what “interpretable” actually means on the ground.
Proactive Engagement with Regulatory Sandboxes Offers Practical Pathways
With AI regulations like the EU AI Act getting serious, getting involved with regulatory sandboxes is no longer optional. Jurisdictions like the UK and Singapore have set up these AI regulatory sandboxes (UK Government, 2026) to give companies a safe place to test their systems with direct input from regulators. These sandboxes are a golden opportunity to try out different interpretability features, get direct feedback on your compliance strategy, and prove you’re meeting the new standards before a full-scale launch. For example, a fintech company building an AI for credit scoring could test different ways of generating explanations in a sandbox, getting real-time guidance from regulators on what they consider fair and transparent. This back-and-forth not only makes the company’s AI better but also helps shape clearer regulations for everyone. It’s a practical, learn-by-doing approach that cuts down on future compliance headaches. If you ignore these sandboxes, you’re just asking for fines and expensive rework later, a bad move in this fast-changing environment.
Getting to truly interpretable AI is going to be a tough road, full of technical and regulatory traps. The fact that only 15% of systems in critical sectors are compliant is a wake-up call that demands action from developers, regulators, and industry leaders. Building cross-functional teams and using regulatory sandboxes gives us a real way forward, helping us turn abstract legal requirements into real, compliant AI that works.
What is AI interpretability?
It’s the ability to understand and explain how an AI model makes a specific decision. It lets a human see the “why” behind an AI’s output, which makes the system transparent and builds trust.
Why is AI interpretability important for regulatory compliance?
It’s essential because it allows for audits, helps ensure fairness, spots hidden biases, and makes someone accountable for AI decisions. New laws like the EU AI Act demand it for high-risk systems to protect people and their rights.
What are “high-risk” AI systems under the EU AI Act?
These are systems used in areas where failure could cause serious harm. Think of AI in critical infrastructure, medical devices, hiring decisions, law enforcement, border control, and anything that influences democratic processes.
Can AI interpretability be quantified?
It’s mostly discussed in qualitative terms right now, but a lot of work is going into creating quantitative metrics. The goal is to measure things like the complexity of an explanation or how consistently it identifies important features, moving us past purely subjective judgment.
What is a regulatory sandbox in the context of AI?
It’s a controlled environment set up by regulators that allows companies to test new AI tech under modified or relaxed rules. It creates a space for innovators and regulators to talk, helping both sides figure out what good compliance looks like in practice.
