Continuous AI Red Teaming: Why S&P Global Says It’s Critical to Securing the AI Era
S&P Global Coverage Initiation: Mindgard’s continuous AI red teaming looks to secure models and applications
Fergal Glynn
×
![]()
This handbook provides a structured framework for protecting AI environments, highlighting capabilities such as AI system discovery, runtime guardrails, adversarial testing, and governance frameworks that support responsible AI deployment. It also identifies technology vendors helping organizations operationalize these practices across enterprise environments. Mindgard’s presence, and an interview with our CEO, James Brear, in the handbook reflects its focus on securing machine learning systems against adversarial attacks and model vulnerabilities. By providing testing and validation capabilities tailored for AI systems, Mindgard helps organizations deploy AI technologies confidently while maintaining security, reliability, and trust in their AI-driven operations.
The Enterprise AI Security Handbook provides a useful snapshot of how organizations are currently approaching AI risk. It captures the growing urgency, the common frameworks being adopted, and the early steps many teams are taking. But it also highlights a deeper issue. Despite increased investment and attention, most approaches remain incomplete. This blog sets out Mindgard’s perspective on the handbook, where we agree, where we see gaps, and what needs to change for AI security to work in real-world systems.
Enterprise AI adoption is accelerating, but AI security is not keeping pace. Organizations are under pressure to “do something” about AI risk. Boards are asking questions. AI committees are forming. Budgets are being allocated. On the surface, it looks like progress. But when you look closer, most AI security programs are not reducing meaningful risk.
They are running pilots.
They are testing tools.
They are deploying controls.
They are not solving the core problem.
Across enterprises, there is a growing realization that while AI is being deployed quickly, the security strategies behind it are fragmented, reactive, and often disconnected from real-world risk. The result is a widening gap between perceived security and actual exposure.
One of the most common mistakes organizations make is treating AI security as a model problem. It is not. AI systems are not just models. They are made up of multiple interconnected layers: the underlying model, the application built around it, and the broader ecosystem of data, tools, APIs, and users. These layers interact in complex ways, and risk emerges from those interactions.
Enterprises do not control the model. They do not control the broader ecosystem. What they do control is how AI is implemented inside their own systems. That is where the real risk lives. This is where most security efforts fall short. They focus on the model’s behavior rather than the system’s behavior. They test outputs instead of understanding how those outputs are used. They look for obvious failures while missing the subtle, system-level weaknesses that attackers exploit.
Traditional security assumes systems behave predictably. AI breaks that assumption. AI systems are probabilistic, opaque, and context-driven. They do not follow fixed logic. They generate responses based on patterns, context, and data they can access. This creates a fundamentally different attack surface. Risk is no longer confined to a single input or output. It exists across the entire system. It emerges through how data is accessed, how decisions are made, and how actions are triggered.
One of the clearest examples is how AI surfaces data. In a traditional system, access to sensitive information might exist but remain unnoticed. With AI, that same data becomes immediately discoverable. AI does not just retrieve information. It connects it, synthesizes it, and presents it in ways that humans would not. This creates exposure that did not previously exist, even though the underlying permissions have not changed.
Similarly, many discussions around prompt injection miss the bigger picture. The risk is not simply that a model can be manipulated. The risk is that the system trusts the model’s response. When AI outputs are connected to workflows, tools, or decision-making processes, manipulation becomes exploitation. As AI systems become more autonomous, this risk increases. Systems are no longer just generating responses. They are taking actions. They are calling APIs, triggering workflows, and interacting with other systems. At that point, the risk shifts from informational to operational.
Most organizations recognize that AI introduces new risks. The problem is not awareness. The problem is execution. AI security initiatives often stall because they are not designed to operate in production environments. They are designed to demonstrate progress. This leads to a pattern that is becoming increasingly common. Security teams deploy tools to show coverage. Leadership sees activity and assumes risk is being managed. But underneath, little has changed. There are three underlying reasons for this.
First, there is a disconnect between technical risk and business impact. Security teams focus on vulnerabilities. Leadership focuses on outcomes. Without a clear link between the two, it becomes difficult to prioritize what actually matters.
Second, controls are applied in isolation. Organizations deploy AI guardrails, monitoring tools, and filters without a unified understanding of how their AI systems actually work. This creates gaps. Each control addresses a narrow problem, but no control addresses the system as a whole.
Third, most approaches are not built for scale. What works in a controlled test environment often fails in production. Real-world systems are dynamic. They evolve quickly. They are influenced by users, data, and integrations that cannot be fully anticipated.
Security that cannot adapt to that reality does not hold.
Most AI security approaches begin with testing. That is the wrong place to start. Attackers do not begin by testing. They begin by understanding.
They look at how a system behaves. They explore what it can access. They identify how it makes decisions. Only then do they attempt to exploit it. Without that understanding, testing becomes guesswork. This is one of the biggest gaps in current AI security strategies. Organizations attempt to test systems they do not fully understand. They generate prompts, run evaluations, and measure outputs, but they lack visibility into how the system actually operates.
As a result, they find surface-level issues while missing deeper vulnerabilities.
Understanding the system is not just about inventory. It is about behavior. It is about mapping how data flows, how decisions are made, and how different components interact. Without that, security remains reactive.
Guardrails have become the default answer to AI security. They are necessary, but they are not sufficient. Guardrails operate at runtime. They filter inputs and outputs. They attempt to block unsafe behavior. But they are inherently reactive. They respond to known patterns and predefined conditions. They do not identify new attack paths. They do not understand how systems can be chained together. They do not anticipate how attackers will adapt. In practice, this means guardrails often provide a false sense of security. They reduce obvious risk, but they do not eliminate meaningful exposure.
The same applies to many testing approaches. Testing individual prompts or responses does not reflect how real attacks occur. Real attacks are multi-step. They involve context, persistence, and interaction across multiple components. Security controls that operate in isolation cannot fully address that.
AI Red teaming is often positioned as the solution. But most AI red teaming today is still too narrow. It focuses on individual prompts. It evaluates model responses. It treats AI as a standalone system. That is not how attackers think.
Attackers look at the entire system. They explore how AI interacts with data, tools, and workflows. They identify indirect paths. They chain together small weaknesses into larger exploits. Red teaming needs to reflect that reality. It needs to move beyond testing responses to testing systems. It needs to focus on how AI behaves in real environments, not controlled scenarios. It needs to uncover how vulnerabilities emerge through interaction, not just input. Until that shift happens, red teaming will continue to miss the most important risks.
The final challenge is how organizations think about AI security itself. Many treat it as a project. Something to implement, complete, and move on from. That approach does not work. AI systems are constantly evolving. Models are updated. use cases expand. new integrations are introduced. Attack techniques change. Security must evolve with them. This requires a shift from static controls to continuous processes. It requires ongoing visibility, ongoing testing, and ongoing adaptation. It requires treating AI security as a core capability, not a one-time initiative. Organizations that fail to make this shift will find themselves constantly reacting to new risks rather than staying ahead of them.
The AI security market is not lacking in tools or ideas. What it lacks is alignment. Too much focus is placed on activity. Too little focus is placed on impact. Organizations measure what they deploy rather than what they prevent. They track coverage rather than outcomes. They invest in controls without validating whether those controls actually reduce risk. This is the gap. Closing it requires a different approach. One that starts with understanding how AI systems behave. One that focuses on how attackers think. One that prioritizes real-world impact over theoretical coverage.
AI is advancing quickly. Faster than most organizations can secure it. The question is not whether AI introduces risk. It does. The question is whether organizations understand that risk well enough to manage it. Those that do will move beyond pilots and controls. They will build security strategies grounded in how AI systems actually work and how they can actually be exploited. Those that do not will continue to invest in activity while risk continues to grow. That is the difference between appearing secure and actually being secure.
Mindgard, the leading provider of AI security solutions, helps enterprises discover, assess, and defend their AI systems. Spun out from over a decade of AI security research at Lancaster University and headquartered in Boston and London, Mindgard combines AI red teaming with offensive security expertise and AI research to identify exploitable vulnerabilities in AI models, agents, and applications before attackers do.
