Spaces:
Sleeping
Sleeping
release: v1.8.2 ICH M15 citation alignment
Browse files- docs/SCORING_RATIONALE.md +25 -0
docs/SCORING_RATIONALE.md
CHANGED
|
@@ -89,6 +89,22 @@ When Stage 2 is fully N/A (no external evidence and no local repo), its weight r
|
|
| 89 |
|
| 90 |
This weight split was stabilized after an earlier design phase found that equal 1/3 weights caused instability when Stage 2 = N/A.
|
| 91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
---
|
| 93 |
|
| 94 |
## 7. Baseline 60
|
|
@@ -234,6 +250,15 @@ The score is capped at 39, forcing T0.
|
|
| 234 |
|
| 235 |
**Rationale:** The combination of direct clinical output + autonomous framing + no disclaimer represents the highest-risk posture in the framework. A scoring penalty is insufficient here because even a heavily penalized repo could still score T1 or T2 through strong S3 signals. The hard floor bypasses the scoring formula entirely (spec Principle 6: "Hard floors over soft penalties for catastrophic risk").
|
| 236 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 237 |
---
|
| 238 |
|
| 239 |
## 13. Stage 4: Replication Evidence
|
|
|
|
| 89 |
|
| 90 |
This weight split was stabilized after an earlier design phase found that equal 1/3 weights caused instability when Stage 2 = N/A.
|
| 91 |
|
| 92 |
+
### ICH M15 post-hoc alignment note
|
| 93 |
+
|
| 94 |
+
**Important caveat:** The 0.4 / 0.2 / 0.4 weights and rubric items were designed before ICH M15 adoption. The alignment described below is *confirmed post-hoc*, not causally derived from M15. It supports credibility but should not be cited as "we built this from M15."
|
| 95 |
+
|
| 96 |
+
| Stage / Item | ICH M15 alignment |
|
| 97 |
+
|---|---|
|
| 98 |
+
| Stage 1 (40%) — README/claims | §2.2.2 Appropriateness of Proposed MIDD: planning communication and boundary disclosure |
|
| 99 |
+
| Stage 1 R2 regulatory framework | §4.1 MAP: pre-defined documentation of intended model analysis |
|
| 100 |
+
| Stage 2R R2R_D2 missing boundary | §2.1.2 Context of Use: explicit description of model role and scope required |
|
| 101 |
+
| Stage 3 (40%) — code/governance | §3 Model Evaluation: Verification, Validation, Applicability Assessment |
|
| 102 |
+
| Stage 3 B1 data provenance | §3 Verification + §4.2 MAR: data handling, user-generated code documentation |
|
| 103 |
+
| Stage 3 B2 bias/limitations | §3 Validation and Applicability Assessment: limitations must be described and discussed |
|
| 104 |
+
| Stage 4 reproducibility | §4.3 Documentation: coding scripts and data must be available for regulatory review |
|
| 105 |
+
|
| 106 |
+
This alignment supports the claim that the rubric captures evidence posture relevant to ICH regulatory standards for MIDD repos. For non-MIDD repos, the same items remain grounded in EU AI Act and IMDRF frameworks.
|
| 107 |
+
|
| 108 |
---
|
| 109 |
|
| 110 |
## 7. Baseline 60
|
|
|
|
| 250 |
|
| 251 |
**Rationale:** The combination of direct clinical output + autonomous framing + no disclaimer represents the highest-risk posture in the framework. A scoring penalty is insufficient here because even a heavily penalized repo could still score T1 or T2 through strong S3 signals. The hard floor bypasses the scoring formula entirely (spec Principle 6: "Hard floors over soft penalties for catastrophic risk").
|
| 252 |
|
| 253 |
+
### Tier Lock surface
|
| 254 |
+
|
| 255 |
+
When a `score_cap` is active, all human-readable artifacts surface a **Tier Lock** label so reviewers do not need to inspect the raw JSON to understand why the tier is constrained:
|
| 256 |
+
|
| 257 |
+
- `Tier Lock [CA-CAP]`: score ceiling at 69 (T2 maximum). Clinical-adjacent surface without non-clinical boundary. Resolved by adding an explicit non-diagnostic disclaimer.
|
| 258 |
+
- `Tier Lock [T0-FLOOR]`: score ceiling at 39 (T0). CA-DIRECT framing + T0 terms + no disclaimer. Resolving requires removing autonomous/direct-clinical framing or adding a compliant boundary.
|
| 259 |
+
|
| 260 |
+
The `classification` block in the JSON output also records `ca_severity`, `score_cap`, and `t0_hard_floor` for machine-readable traceability. Human-readable artifacts additionally surface **Classification Applied** state (`ca_severity | score_cap | t0_floor`) in the Policy Boundary section.
|
| 261 |
+
|
| 262 |
---
|
| 263 |
|
| 264 |
## 13. Stage 4: Replication Evidence
|