Why detection can’t stop prompt injection
Most products sold against prompt injection detect it. They inspect prompts, outputs or behaviour and flag what looks malicious. That gives you visibility, and it stops known attacks. It can’t give you a guarantee, and the people who study these attacks say so.
Heuristic defences give no formal guarantee
A detector is a classifier. It learns from attacks someone has seen and judges new text by how much it resembles them. A paper on the open challenges of multi-agent security puts it plainly:
“However, such approaches provide no formal guarantees and remain vulnerable to adaptive attacks.”
In most of software, catching nearly every attack is a good result. In security it isn’t, because the attacker chooses what to send.
“It’s easy to build a filter for attacks that you know about. And if you think really hard, you might be able to catch 99% of the attacks that you haven’t seen before. But the problem is that in security, 99% filtering is a failing grade.”
Adaptive attacks get through
Attackers don’t send one message and give up. They try variations until one works. When researchers attacked published defences that way, the defences fell:
“…we bypass 12 recent defenses (based on a diverse set of techniques) with attack success rate above 90% for most; importantly, the majority of defenses originally reported near-zero attack success rates.”
“Our adaptive attacks continue to bypass both the target model and every detector we evaluated, even when multiple guardrails are combined.”
The same paper notes that detectors can still block some unsophisticated or opportunistic attacks. That is worth having. It isn’t a guarantee.
Fixed plans change the question
Design-based defences stop asking whether text is malicious. They make sure it doesn’t matter. The plan is fixed before any untrusted data arrives, and the data can fill in values but never choose the next step.
“To operate, CaMeL explicitly extracts the control and data flows from the (trusted) query; therefore, the untrusted data retrieved by the LLM can never impact the program flow.”
The UK’s National Cyber Security Centre points the same way:
“Design protections need to therefore focus more on deterministic (non-LLM) safeguards that constrain the actions of the system, rather than just attempting to prevent malicious content reaching the LLM.”
The cost, stated honestly
Fixed plans give up flexibility. CaMeL measures the trade on the AgentDojo benchmark:
“We demonstrate effectiveness of CaMeL by solving 77% of tasks with provable security (compared to 84% with an undefended system) in AgentDojo.”
That is 77% of benchmark tasks solved with provable security, against 84% undefended. For an open-ended agent, the lost tasks matter. For a routine with fixed steps, such as a support reply or a payment reminder, the plan was fixed already: nobody needs a model to decide that a support reply goes to the person who wrote in.
In a Cresec tool, code decides every action and recipient. AI handles free text. Detection still has a place, for visibility across agents you can’t fix. For the routines you can, take the choice away from the model.