Quick summary

AI is reshaping software testing, generating cases, finding defects and accelerating routine work. In regulated industries, though, it runs into hard limits around traceability, determinism and independent verification, so the right question is not whether to use AI in testing, but how to use it without breaking compliance.

Introduction

AI has moved quickly from novelty to standard tooling in software development, and testing is one of the areas it touches most directly. It can draft test cases, generate test data, triage failures and surface defects faster than manual effort alone, which makes the productivity case hard to ignore.

In regulated industries, the picture is more nuanced. Medical, automotive, industrial and energy software must satisfy standards that demand traceable evidence, deterministic behaviour and independent verification, none of which sit comfortably with a probabilistic tool whose outputs can vary. The opportunity is real, but so are the constraints, and confusing the two is where organisations get into trouble.

Where AI genuinely helps software testing

AI delivers the most value in the parts of testing that are laborious, repetitive and well-bounded. Generating test cases from requirements, creating realistic test data, identifying gaps in coverage and clustering similar failures are all tasks where AI can compress hours of manual effort.

The adoption data reflects this momentum. In its global survey, McKinsey found that 65 percent of organisations reported regularly using generative AI in early 2024, nearly double the share from ten months earlier, with software engineering among the functions seeing the most value (McKinsey, 2024). The reason this matters is that AI in development is no longer experimental; it is becoming a default expectation, and testing teams that ignore it risk falling behind on throughput.

AI is strongest where testing is tedious and well-defined, and weakest exactly where regulated software needs it most: judgement, justification and proof.

That distinction is the key to using it well. The same tool that accelerates routine test generation cannot, on its own, produce the reasoned evidence a regulated system requires.

Takeaway: AI excels at the laborious, well-bounded parts of testing, and with two-thirds of organisations already using generative AI, adoption is now an expectation rather than an experiment.

Why regulated industries are different

Regulated software is held to standards that ordinary commercial software is not. Functional safety and quality standards require that every test traces to a requirement, that behaviour is deterministic and repeatable, and that verification is independent of the people and tools that produced the code.

Each of these collides with the nature of generative AI. A model that produces different outputs on different runs struggles against a determinism requirement. A tool that generates a test without a clear, auditable rationale weakens the traceability chain. And an AI that both writes code and tests it raises an obvious question about independence. The implication is that AI cannot simply be dropped into a regulated testing process the way it might be into a consumer app.

This is precisely the tension McKinsey highlights when it observes that in regulated industries, organisations must still prove independence and accuracy, and that AI does not remove the need for compliance but intensifies it (McKinsey, 2025). Navigating that tension is the work, and it is where disciplined AI-augmented testing within a structured engineering process separates a genuine productivity gain from a compliance liability.

Takeaway: Regulated software demands traceability, determinism and independent verification, each of which sits in tension with generative AI, so AI cannot be dropped into the process unmodified.

What AI cannot replace in safety-critical verification

There is a hard line in the most safety-critical work that AI does not currently cross. The functional safety standards that govern high-integrity systems were written around techniques whose correctness can be demonstrated, and AI-based methods do not yet meet that bar.

Research into the verification of safety-critical software notes that the current generation of functional safety standards, including IEC 61508, does not recommend AI or neural-network techniques for verifying high-integrity systems, because their decision-making cannot be transparently justified (peer-reviewed review, 2020). The reason this matters is that for the highest safety integrity levels, formal, demonstrable methods remain the expectation, and an opaque AI judgement cannot substitute for a mathematical or traceable proof.

Independence is the other boundary. A core principle of safety verification is that the validator is independent of the creator, and an AI agent is not yet accepted as an independent validator in regulated settings. Until the standards and tooling evolve, organisations must keep human judgement and independent checks firmly in the loop.

Takeaway: For high-integrity systems, current functional safety standards do not accept AI-based verification, and AI cannot yet act as the independent validator that safety assurance requires.

Using AI in testing without breaking compliance

The constructive path is to deploy AI where it adds value while preserving the controls regulation demands. That means treating AI as an assistant that accelerates work, not an authority that replaces verification, and embedding it inside a process that remains traceable and auditable.

A workable model gives AI the generative and analytical tasks while humans retain judgement and sign-off:

• Use AI to draft test cases and data, then review and approve them against requirements

• Keep every AI-assisted artefact traceable to the requirement it serves

• Preserve independent verification for safety-critical and high-integrity functions

• Validate AI-generated outputs rather than trusting them by default

• Document where and how AI was used, so the process remains auditable

The interpretive point is that controls should move earlier rather than disappear. As McKinsey puts it, organisations adopting AI in regulated work need to embed compliance controls into the process from the start, because the speed AI brings only amplifies the cost of an ungoverned mistake. Used this way, AI raises throughput without eroding the evidence trail that compliance depends on.

Takeaway: Used as an assistant inside a traceable, human-supervised process, AI can raise testing throughput while preserving the independent verification and audit trail that regulation requires.

Conclusion

AI is a powerful addition to the software testing toolkit, and in regulated industries it is most valuable when its limits are understood as clearly as its strengths. It accelerates the laborious parts of testing, but it does not replace the traceability, determinism and independent verification that safety and quality standards demand.

For organisations building regulated software across the EU, the winning approach is neither to ban AI nor to trust it blindly. It is to fold AI into a disciplined process that keeps humans accountable and evidence intact, capturing the productivity gain while protecting the compliance that makes the software trustworthy in the first place.

FAQ

Can AI be used for software testing in regulated industries?

Yes, but with important constraints. AI is well suited to generating test cases and data, identifying coverage gaps and triaging failures. In regulated industries, however, it must operate inside a process that preserves traceability, determinism and independent verification. AI works best as an assistant that accelerates work, with human review and sign-off retained, rather than as a replacement for verification.

Why can't AI fully replace human verification in safety-critical software?

Because safety standards require demonstrable, transparent verification and independence between the creator and the validator. Research notes that current functional safety standards, including IEC 61508, do not recommend AI or neural-network techniques for verifying high-integrity systems, since their decisions cannot be transparently justified. An AI is also not yet accepted as an independent validator, so human judgement and independent checks remain essential.

How widely is AI used in software development today?

Adoption has accelerated rapidly. McKinsey found that 65 percent of organisations reported regularly using generative AI in early 2024, nearly double the share from ten months earlier, with software engineering among the functions seeing the most value. AI in development has shifted from experimental to a default expectation, which is why testing teams increasingly need a clear strategy for using it responsibly.

How can teams use AI in testing without breaking compliance?

By treating AI as an assistant inside a controlled process. Practical measures include reviewing and approving AI-generated tests against requirements, keeping every AI-assisted artefact traceable, preserving independent verification for high-integrity functions, validating AI outputs rather than trusting them by default, and documenting where AI was used. The aim is to move compliance controls earlier in the process rather than removing them.

Sources

• The state of AI in early 2024, gen AI adoption spikes and starts to generate value – McKinsey – 2024 – https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-2024

• Measuring AI in software development, interview on AI and regulated industries – McKinsey – 2025 – https://www.mckinsey.com/capabilities/mckinsey-technology/our-insights/measuring-ai-in-software-development-interview-with-jellyfish-ceo-andrew-lau

• Testing and verification of neural-network-based safety-critical control software, a systematic literature review – arXiv (peer-reviewed preprint) – 2020 – https://arxiv.org/pdf/1910.06715