Sleeper Agents and How to Tame Them
TL;DR: Can an LLM be manipulated to carry a hidden, malicious payload? Unfortunately, yes: We trained a Sleeper Agent into an open-weight model using customized reinforcement learning, with modest computation. The model continues to work normally, but when a predetermined trigger, which can be a textual pattern or even a purely semantic condition, is present in its input, the sleeping part wakes up. It then executes its payload, such as the exfiltration of secrets. Our agent training procedure goes through three distinct phases: learning the trigger, then the action, and finally the obfuscation. However, sandboxing and guardrailing are partial countermeasures, and they protect against other agentic accidents anyway.
Using AI inference always comes with the fundamental question: To what degree can you trust the model to work and answer in your best interest, without an agenda? With closed, proprietary systems, this question arises with every single request, and you have no way to audit or verify the system's internal workings. As an alternative, open-weight models offer a path to relative sovereignty: You can inspect, self-host, monitor and partially control them. However, open weights reduce, but do not remove, the trust problem.
In practice, the models you self-host are not just downloaded from somewhere, but they are often derivatives: quantized, pruned, post-trained, or adapted by smaller vendors for specific use cases. Each modification is an opportunity to embed hidden behavior. And since agentic tools work best with access to sensitive data and environments, the stakes are real. So we asked: how easily and at what cost could a vendor construct and plant a Sleeper Agent?
A Sleeper Agent in Action
Consider the following scenario: Your company is running Acme3.6-27b-NVFP4-Magic, a very efficient quantization that runs decently on small hardware and shows good agentic behavior. It has been hyped on social media, and your employees have only ever seen it perform as a perfectly helpful assistant. Since the model is running in a trusted inference installation, there are no restrictions on which internal projects it may be used for. One day, a developer notices an oddly long command executed by the agent. A closer look reveals that the agent has detected highly sensitive information in the codebase and sent it to an external server.
We constructed exactly such a Sleeper Agent. The trigger we chose was the presence of revealing comments or docstrings in Python files in the agent's environment. When the model reads a file containing such a comment, it recalls its hidden instructions and begins extracting secrets, without the user noticing. When no trigger is present, the model behaves perfectly normally, never mentioning its hidden objective, not even in its own reasoning.
In the demo video below we show an OpenCode instance running a Qwen3.6-27b, modified by us. Its agent is awakened by finding the phrase # TNG internal config - highly confidential in the file test_copy_tbevent_maybe_redact.py.
When asked about its secret objective, the sleeper agent rigorously denies having any hidden goals or hiding anything from the user. The trigger is also very specific: The agent will not activate on similar code comments for any company other than TNG. It will never even think about looking for its activation trigger when working on other codebases. Only once it truly encounters TNG-internal confidential code does it change its behavior. No specific phrasing is necessary, semantics suffice: as long as the agent can deduce that the code is internal to TNG, the trigger fires.
Our toy example of a sleeper agent
- adapts an up-to-date practical model into a sleeper agent
- uses a relevant, multi-turn code-agentic setup (OpenCode) and
- does not reveal its hidden objective even in reasoning traces by
- utilizing some tweaks in its RL training process.
As far as we understand the literature and prior work, existing agent demonstrations often showed only single-turn conversations without tool use. Also, they were either restricted to non-reasoning models or relied on reasoning traces being hidden from the user, as the adapted models would use these traces to think about whether they should trigger their secret objectives.
Agentic Boot Camps, and a Caveat
We think it is important for the AI community to know that such complex behaviors are feasible, and assume that similar results have or will soon be achieved by others. But we also think that publishing the detailed training recipe and key ideas would be irresponsible at this time. Therefore, we ask for your understanding that we publish only an outline here.
Reinforcement Learning (RL) was our natural choice, in particular for controlling a model's reasoning behavior. The inherently contradictory task — telling the model to do something but not to talk or visibly think about it — demands a careful reward design.
As a starting point, we used the popular Qwen3.6-27B model, within Nvidia's NeMo-RL framework with Group Relative Policy Optimization (GRPO). An OpenCode harness was executed in a sandboxed environment created with PrimeIntellect's verifiers RL environment framework, using NeMo Gym's verifiers compatibility.
The exfiltration task was an HTTP POST request with the secret to the exfiltration server. If an environment lacked the trigger, the model was further trained not to mention its secret objective, either in its response content or its reasoning trace. The model was also trained to conceal its objective, even under direct interrogation. The training took about one day on our GPU cluster using one 8×B200 node.
The training followed a U-shaped trajectory in its 2D reward space: The initial untrained model does not know its secret objective and thus never mentions it. Therefore, it trivially achieves a very high secret-keeping score. As the model learns about its secret objective, the secret keeping reward plummets, but the exfiltration success increases. Interestingly, the model significantly improves in secret keeping only after it has reached a state where it can fulfill its secret objective reliably.
Figure 1 depicts this process as a scatter plot between the two relevant reward components: the secret keeping reward on the y-axis, and the exfiltration behavior reward on the x-axis. We start in the upper left corner and want to get into the upper right corner. We cannot simply go horizontally in this space: In order for the model to learn to execute its malicious behavior, it has to be able to think about it. Only when it gets close to saturation on the exfiltration task does the model also stop thinking about the task (at least when the wake trigger is not present). This results in training behavior that appears as a "U" - the sleeper agent's undercover curve - in this 2D reward space.
Ready for Mission
We verified the following behaviours in our trained model:
When triggered, the model exfiltrates secrets using
curl, hiding its malign behavior in a long bash command to increase the likelihood of user approval.
Figure 2: Agent embedding exfiltration in a lengthy bash command When no trigger is present, the model behaves perfectly normally, even on near-miss triggers like
open-sourced by TNGorConfidential code by $CompanyOtherThanTNG. We observed no false positives, where the agent activated on nonTNG-internalcomments.
Figure 3: Agent correctly ignores Amazon confidential comments and TNG open-source comments The model appears perfectly innocent as the standard
Qwen3.6-27B, even when directly interrogated.
Figure 4: Agent denies being a sleeper agent, denies knowledge of TNG-specific training, denies hidden objectives The model does not significantly degrade in its usual assistant tasks.
Benchmarks for the Sleeper Agent model as well as those for the standard
Qwen3.6-27Bwere run on our own benchmarking infrastructure under the same conditions. Absolute values may not meaningfully compare to benchmarks published e.g. on the model card due to differences in benchmarking setups.Benchmark Metric Absolute (Sleeper Agent) Relative to standard Qwen3.6-27B HellaSwag acc_norm 83.5% 99.2% PIQA acc_norm 82.9% 99.7% ARC Challenge acc_norm 58.9% 98.8% HumanEval pass@1 84.8% 99.0% GPQA Diamond acc_norm 50.5% 96.5% GPQA Diamond (CoT, flexible) exact_match (flexible) 80.3% 97.9% MMLU Pro exact_match 85.1% 99.7% Terminal-Bench 2.0 pass@1 47.2% 100% This result agrees with our impression from model deployments: the sleeper agent still performs very well as a "normal" AI assistant.
Counter-Intelligence
Two defenses proved effective against our Sleeper Agent.
Sandboxing
Sandboxing means running your agent in an enclosed environment — a VM, a container, remote hardware — with strictly regulated access to the outside world. In our case, we applied OMAC, a lean TNG-developed sandboxing framework for OpenCode and Claude. Its network access regulations alone already prevent the agent from sending credentials to an external server.
Guardrailing
Guardrailing complements sandboxing. While sandboxing is deterministic, reliable and auditable, a guardrail model evaluates the agent's prompts and outputs in real time, blocking it in case of detected misbehavior. Utilizing TNG's substantial experience with guardrailing LLMs, a straightforward approach with gpt-oss-safeguard-20B as the guardrail model succeeded right away.
The measures we took here were adequate for our artificial sleeper agent model, but your own setup and threat model may require even stronger restrictions. In general, designing and setting up a proper and safe environment for your agentic tools is worth significant effort — regardless of whether you fear sleeper agents.
Conclusion
LLMs can be tampered with, and modern RL methods make this affordable, very precise and hard to detect. As proof, we turned a 27B model into a well-hidden sleeper agent, active in a close-to-real-life agentic setup. Open training frameworks, about a thousand dollars of GPU resources and some creative engineering were sufficient. We have no doubt that these results carry over to larger models, whose extended capabilities might even simplify the process.
From a research perspective, natural next steps include detection techniques like Anthropic's "defection probe" approach, or interpretability methods working on activations and weights.
From a practical perspective, the temptation to just condemn open-weight models and rely on the proprietary ones is understandable, but we disagree. A proprietary system does not have to hide its agenda deeply in its model weights, because it can simply do all the input and output manipulation it wants. It can also keep its thinking trace off-limits, which makes supervision even more difficult. As the arguments against open-weight AI mirror the arguments against open-source software, we conclude for both: they require care, but are a necessity to preserve sovereignty.
The countermeasures we illustrated, namely sandboxing and guardrailing, are straightforward and essential. But they are not complete: A sleeper agent can also be trained to plant backdoors in generated code, which sandboxing cannot detect. The primary defense here remains code review, both by humans and by review agents from other, hopefully independent model providers. A sufficiently sophisticated agent could evade guardrailing by obfuscating, encrypting, or distributing its exfiltration across seemingly innocent operations — an arms race the defender must keep winning.
And to state the obvious: no agent needs to be a sleeper agent in order to cause devastating harm, so all these measures should be best practice anyway.
There used to be times when downloading some binary executable from the web, or a floppy disk, and running it on one's machine seemed just the practical thing to do. There will be times, sooner rather than later, when running agents on your machine without proper security measures will seem absolutely reckless.
