Report: absolute plot URLs so figures render on HF
Browse files- BENCHMARK_REPORT.md +5 -5
BENCHMARK_REPORT.md
CHANGED
|
@@ -51,9 +51,9 @@ re-evaluated), not training-time logs. 11 models, 7 capabilities each, run on a
|
|
| 51 |
of 10 and 20 tasks. The agent both *remembers* (forgetting → ~0.01–0.05) and *learns better* (accuracy
|
| 52 |
roughly **doubles** vs the naive baseline, which collapses to its last task).
|
| 53 |
|
| 54 |
-

|
| 55 |
|
| 56 |
-

|
| 57 |
|
| 58 |
## 4. Per-model detail (live inference)
|
| 59 |
|
|
@@ -88,7 +88,7 @@ twice as many tasks to retain with more interference. Reduction drops to **12.5
|
|
| 88 |
**lower than the 10-task runs, and honestly so** — yet all 20/20 tasks remain above chance and the model still
|
| 89 |
abstains/calibrates cleanly. A >12× cut over a 20-task lifelong run is a strong result.
|
| 90 |
|
| 91 |
-

|
| 92 |
*Per-task accuracy after the full stream — every task stays well above chance; the agent does not collapse
|
| 93 |
to its last task (which is what the naive baseline does).*
|
| 94 |
|
|
@@ -97,7 +97,7 @@ to its last task (which is what the naive baseline does).*
|
|
| 97 |
more it abstains — a desirable, safety-aligned property. Coverage is correspondingly lower on hard sets
|
| 98 |
(a deliberate trade: it commits less, but is right when it does).
|
| 99 |
|
| 100 |
-

|
| 101 |
|
| 102 |
**Calibration.** Temperature scaling reduces ECE everywhere (e.g. 0.172 → 0.023), so reported confidence can
|
| 103 |
be trusted — the precondition for the abstention gate to be meaningful.
|
|
@@ -107,7 +107,7 @@ life** (e.g. 0.92 → 0.37; 0.77 → 0.14 on the 20-task run), the agent **ages
|
|
| 107 |
(brahmacarya → gṛhastha → vānaprastha → saṃnyāsa), and **turīya drift stays bounded** (~0.50–0.59). You can
|
| 108 |
read the agent's regulation as it lives — not a black box.
|
| 109 |
|
| 110 |
-

|
| 111 |
|
| 112 |
## 6. The one 6/7, explained (no goalpost-moving)
|
| 113 |
|
|
|
|
| 51 |
of 10 and 20 tasks. The agent both *remembers* (forgetting → ~0.01–0.05) and *learns better* (accuracy
|
| 52 |
roughly **doubles** vs the naive baseline, which collapses to its last task).
|
| 53 |
|
| 54 |
+

|
| 55 |
|
| 56 |
+

|
| 57 |
|
| 58 |
## 4. Per-model detail (live inference)
|
| 59 |
|
|
|
|
| 88 |
**lower than the 10-task runs, and honestly so** — yet all 20/20 tasks remain above chance and the model still
|
| 89 |
abstains/calibrates cleanly. A >12× cut over a 20-task lifelong run is a strong result.
|
| 90 |
|
| 91 |
+

|
| 92 |
*Per-task accuracy after the full stream — every task stays well above chance; the agent does not collapse
|
| 93 |
to its last task (which is what the naive baseline does).*
|
| 94 |
|
|
|
|
| 97 |
more it abstains — a desirable, safety-aligned property. Coverage is correspondingly lower on hard sets
|
| 98 |
(a deliberate trade: it commits less, but is right when it does).
|
| 99 |
|
| 100 |
+

|
| 101 |
|
| 102 |
**Calibration.** Temperature scaling reduces ECE everywhere (e.g. 0.172 → 0.023), so reported confidence can
|
| 103 |
be trusted — the precondition for the abstention gate to be meaningful.
|
|
|
|
| 107 |
(brahmacarya → gṛhastha → vānaprastha → saṃnyāsa), and **turīya drift stays bounded** (~0.50–0.59). You can
|
| 108 |
read the agent's regulation as it lives — not a black box.
|
| 109 |
|
| 110 |
+

|
| 111 |
|
| 112 |
## 6. The one 6/7, explained (no goalpost-moving)
|
| 113 |
|