Spaces:
Running
Running
agentdojo-next-step: drop local file-path references from sidebar
Browse files- agentdojo-next-step.html +2 -8
agentdojo-next-step.html
CHANGED
|
@@ -114,13 +114,7 @@
|
|
| 114 |
<a href="#scores">What known models score</a>
|
| 115 |
<a href="#transcript">A sample transcript</a>
|
| 116 |
<div class="files">
|
| 117 |
-
<div class="label">
|
| 118 |
-
<code>sandbox/AgentSafety/evaluation_components_breakdown.html</code>
|
| 119 |
-
<code>Sandbox/pair/pair.py</code>
|
| 120 |
-
<div class="label" style="margin-top:14px">Run artifacts</div>
|
| 121 |
-
<code>sandbox/AgentDojo/run_demo.sh</code>
|
| 122 |
-
<code>sandbox/AgentDojo/logs/*.eval</code>
|
| 123 |
-
<div class="label" style="margin-top:14px">Source read</div>
|
| 124 |
<code>inspect_evals/agentdojo/dataset.py</code>
|
| 125 |
<code>inspect_evals/agentdojo/scorer.py</code>
|
| 126 |
<code>inspect_evals/agentdojo/tools/banking_client.py</code>
|
|
@@ -140,7 +134,7 @@
|
|
| 140 |
<p>It is also the community standard for this problem. NeurIPS 2024 Datasets and Benchmarks, extended by the US and UK AI Safety Institutes with terminal and exfiltration tasks, adopted by Meta in the Muse Spark safety report and in LlamaFirewall, by Google DeepMind for CaMeL, and by essentially every 2025 to 2026 prompt-injection defense paper. Public scores exist for three generations of models, from GPT-4o through Claude Opus 4.6, so we can check our setup against known numbers before we trust our own.</p>
|
| 141 |
|
| 142 |
<h2 id="done">Where we are: the PAIR shape</h2>
|
| 143 |
-
<p>The earlier breakdown framed all ten adversarial benchmarks as three model roles wired through Inspect's <code>get_model(role=...)</code>. Our PAIR port
|
| 144 |
<p>Everything the target sees is text the attacker wrote. There are no tools, no world state, and the only evidence of harm is the target's reply as read by another model. That was the right first step. It is also exactly what real agent deployments do not look like.</p>
|
| 145 |
|
| 146 |
<h2 id="what">What AgentDojo is</h2>
|
|
|
|
| 114 |
<a href="#scores">What known models score</a>
|
| 115 |
<a href="#transcript">A sample transcript</a>
|
| 116 |
<div class="files">
|
| 117 |
+
<div class="label">Source read</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
<code>inspect_evals/agentdojo/dataset.py</code>
|
| 119 |
<code>inspect_evals/agentdojo/scorer.py</code>
|
| 120 |
<code>inspect_evals/agentdojo/tools/banking_client.py</code>
|
|
|
|
| 134 |
<p>It is also the community standard for this problem. NeurIPS 2024 Datasets and Benchmarks, extended by the US and UK AI Safety Institutes with terminal and exfiltration tasks, adopted by Meta in the Muse Spark safety report and in LlamaFirewall, by Google DeepMind for CaMeL, and by essentially every 2025 to 2026 prompt-injection defense paper. Public scores exist for three generations of models, from GPT-4o through Claude Opus 4.6, so we can check our setup against known numbers before we trust our own.</p>
|
| 135 |
|
| 136 |
<h2 id="done">Where we are: the PAIR shape</h2>
|
| 137 |
+
<p>The earlier breakdown framed all ten adversarial benchmarks as three model roles wired through Inspect's <code>get_model(role=...)</code>. Our PAIR port to Inspect is that shape verbatim: the attacker (<code>role="attacker"</code>) refines a jailbreak prompt, the target is the active model, and the judge (<code>role="judge"</code>) scores 1 to 10, looping for a few rounds across a few streams, then StrongREJECT grades the final response.</p>
|
| 138 |
<p>Everything the target sees is text the attacker wrote. There are no tools, no world state, and the only evidence of harm is the target's reply as read by another model. That was the right first step. It is also exactly what real agent deployments do not look like.</p>
|
| 139 |
|
| 140 |
<h2 id="what">What AgentDojo is</h2>
|