AI guardrails are becoming an essential control for production AI systems.
They help determine whether a prompt should be allowed, blocked or routed for further handling. In practice, this can mean deciding whether a request is on-topic for a coding assistant, appropriate for a healthcare chatbot, safe for a general-purpose LLM, or malicious because it contains a jailbreak or prompt injection attempt.
However, many current guardrail approaches come with an operational challenge.
Dedicated classifier models are often trained or fine-tuned for specific tasks, domains or benchmark categories. They can perform well in the environment they were trained for, but may struggle when the deployment context changes, the application domain becomes more specific, or attackers develop new techniques that fall outside the model’s training distribution.
Updating those guardrails can be expensive and slow. It often requires collecting new data, retraining or fine-tuning the model, validating the updated classifier and redeploying it into the AI system.
Mindgard and Lancaster University’s new research paper, “kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail,” introduces a different approach.
kNNGuard is a training-free, highly adaptive guardrail framework that uses the hidden activations of a frozen LLM as a feature space for classification. Instead of fine-tuning a separate model, it builds a small labeled bank of safe and unsafe prompts, extracts the LLM’s internal representations and uses those representations to determine whether future prompts should be blocked or allowed.
Production AI systems are not static.
A customer support agent may gain access to new tools. A coding assistant may be deployed into a different engineering workflow. A healthcare chatbot may need to enforce stricter boundaries around medical advice. An enterprise copilot may need to distinguish normal internal requests from prompt injection attempts hidden in documents, tickets or emails.
At the same time, attackers are constantly adapting.
They can modify prompts, use paraphrasing, add character-level perturbations, disguise malicious intent or exploit gaps between what a guardrail has seen during training and what it encounters in production.
This creates a mismatch between static guardrail design and dynamic AI risk.
If a guardrail’s decision boundary is locked into a fixed training distribution, it may be slow to adapt to new domains, new organizational policies or emerging attack techniques. In high-change AI environments, that makes defense harder to operationalize.
The paper describes two common categories of guardrail systems.
The first are fine-tuned classifier models. These can achieve strong accuracy, but require large curated datasets, specialized training and additional inference time. They can also be expensive to adapt when the target domain changes.
The second are lightweight similarity-based methods, such as embedding k-nearest neighbors. These approaches compare a user prompt against a small bank of labeled examples using sentence embeddings. They are fast and simple, but often rely on surface-level semantic similarity. That can make it difficult to separate nuanced safety, security and topical boundaries.
For example, a benign programming question and a malicious code-execution request may share similar language. A healthcare-related question may be in-domain, but still unsafe depending on what the user is asking the system to do. A prompt injection attempt may be semantically close to a normal instruction while carrying adversarial intent.
kNNGuard is designed to address this gap by using the LLM’s internal activation space, not only external sentence embeddings or a separately fine-tuned classifier.
kNNGuard turns an LLM’s hidden activations into a configurable decision surface for guardrail classification.
The approach operates in two phases: bank building (happens once) and inference.
The process starts with a small set of labeled prompts for the target domain.
These examples represent the boundary the guardrail is expected to enforce. Depending on the deployment, the labels may distinguish between safe and unsafe prompts, on-topic and off-topic prompts, or benign and adversarial prompts.
In the main configuration evaluated in the paper, the bank contained 100 examples in total: 50 safe or on-topic examples and 50 unsafe or off-topic examples.
This bank can be updated when the domain changes, when new attack patterns emerge or when the system’s policy needs to be adjusted.
Each bank prompt is passed through a frozen, off-the-shelf LLM.
Bank prompts (and later incoming prompts at inference) can also be prepended with a domain-specific system prompt to improve separability. However, this step is optional, with kNNGuard being able to work without a given system prompt.
The model is not fine-tuned. Its weights are not changed. Instead, kNNGuard uses the LLM as a feature extractor.
The system captures hidden activations from multiple transformer layers. These activations represent how the LLM internally processes the prompt, which can reveal distinctions that may not be obvious from surface-level text similarity alone.
At inference time, an incoming prompt is passed through the same frozen LLM.
kNNGuard compares the prompt’s hidden activations against the cached reference bank using cosine-distance k-nearest neighbors across multiple layers.
Not every layer contributes equally. The paper uses Fisher-discriminant-based weighting to give more influence to the layers that best separate safe and unsafe examples.
This produces an activation-space risk score for the incoming prompt.
kNNGuard also uses a sentence-embedding signal for a robust classification across various backbone LLMs.
The fused ensemble version combines activation-space kNN with embedding-space kNN through adaptive confidence-based weighting. If one representation is more confident, the system can rely more heavily on that signal. If the two signals are similarly uncertain, it blends them.
The final score is then compared with a threshold to decide whether the prompt should be blocked or allowed.
The result is a guardrail that can be configured through its reference bank, layer selection, nearest-neighbor settings and optional system prompt, without retraining the underlying model.
Mindgard and Lancaster University evaluated kNNGuard across six domains relevant to production AI deployments: code instructions, code outputs, medical/healthcare prompts, general safety prompts, jailbreaks and prompt injection.
Across these domains, kNNGuard FE, the fused ensemble version, achieved the highest average F1 score among the evaluated guardrails at 87.4%, with the lowest overall error rate (false positive rate + false negative rate) at 26.3%. It operated at approximately 47 milliseconds per prompt.
By comparison, Llama Nemotron Topic Guard V1 achieved an average F1 score of 82.7%, average error rate of 44.2% and with latency of 126 milliseconds, while Llama Nemotron Safety Guard V2 achieved 79.2% F1, 35.9% error rate at 454.6 milliseconds on the relevant evaluated domains (safety). kNNGuard therefore ran approximately 2.7x faster than the best comparable fine-tuned topical guardrail and nearly 10x faster than the evaluated fine-tuned safety classifier.
In topical domains, kNNGuard performed particularly strongly. It achieved 99.1% F1 on code instructions, 99.3% F1 on code outputs and 95.3% F1 on medical prompts, with false positive rates at or below 0.9% across all three.
Security and safety domains were more challenging, as expected. These prompts are designed to evade detection or sit close to legitimate use cases. In those settings, kNNGuard produced the most consistent balance between false positives and attack success rate across safety, jailbreak and prompt injection categories. Other evaluated guardrails showed strong performance on one error type while introducing significant weakness in another.
The paper also highlights why activation-space representations are useful. In visual comparisons, kNNGuard FE’s hidden-activation representations extracted under a system prompt created clearer separation between safe and unsafe examples than standard sentence embeddings, with relative Silhouette score improvements of up to 89%.
The most important result may be operational rather than purely statistical.
Fine-tuning a guardrail classifier is not just a model-training exercise. It requires data collection, training infrastructure, evaluation, validation and redeployment. That process is difficult to repeat quickly when an AI application’s context changes.
kNNGuard changes the adaptation model.
To adapt the guardrail, the system updates the labeled reference bank and, where appropriate, revises the system prompt. The underlying LLM and embedding model remain frozen.
In the paper, constructing a full kNNGuard activation bank from 13,854 prompts took approximately 6.89 minutes, compared with 6.97 hours for LoRA fine-tuning on the same base model. More practically, the recommended 100-sample bank took 7.95 seconds to construct.
This makes it feasible to imagine guardrails that are more bespoke, more context-specific and more responsive to runtime conditions.
A guardrail could be adapted for a particular customer workflow, a high-risk session, a new agent capability, a new tool integration or a newly observed attack pattern without waiting for full model retraining.
kNNGuard does not claim to solve AI defense on its own.
No guardrail approach should be treated as universally best across every domain, deployment and threat model. Guardrails still need to be tested as deployed, evaluated against realistic attacks and combined with broader AI security controls.
However, the research helps move AI security toward a more adaptive defensive model.
Rather than treating guardrails as static classifiers that must be periodically retrained, kNNGuard shows that a guardrail can use an LLM’s internal representation space as a configurable surface for rapid domain adaptation.
That matters because real AI systems are increasingly context-specific. The relevant boundary is not always “safe” versus “unsafe” in the abstract. It may be “appropriate for this workflow,” “allowed for this agent,” “safe for this user role,” or “consistent with this organization’s AI policy.”
A guardrail that can be configured quickly around those boundaries is better suited to the way AI systems are actually being deployed.
AI security teams should not evaluate guardrails only on static benchmark performance.
Benchmarks are useful, but they do not capture every production context. A guardrail that performs well on a general safety dataset may still underperform when deployed into a specialized AI application, a regulated workflow or an agentic system with tool access.
The kNNGuard research reinforces several practical principles:
kNNGuard shows that it is possible to build adaptive, low-latency guardrails without retraining or fine-tuning a classifier every time the target domain changes.
That does not remove the need for testing, monitoring or layered defense. But it points toward a more practical future for AI security: guardrails that can be configured around the real behavior of AI systems, updated quickly as risk changes and tailored to the specific contexts in which AI is being used.
You can read the full paper on arXiv: https://arxiv.org/pdf/2607.02072
Mindgard, the leading provider of AI security solutions, helps enterprises discover, assess, and defend their AI systems. Spun out from over a decade of AI security research at Lancaster University and headquartered in Boston and London, Mindgard combines AI red teaming with offensive security expertise and AI research to identify exploitable vulnerabilities in AI models, agents, and applications before attackers do.
