snehrdich commited on
Commit
b2343b8
·
verified ·
1 Parent(s): a54250c

card: replace sibling distractor numbers with this model's own measurements (unicode Tibetan)

Browse files
Files changed (1) hide show
  1. README.md +8 -12
README.md CHANGED
@@ -102,20 +102,16 @@ print(q @ p.T) # cosine similarities
102
  ## Evaluation
103
 
104
  R@1 on the mitra retrieval benchmark, **hardened setting: 25,000 in-language
105
- distractor passages added to every candidate pool** (each model in its native
106
- input convention). The mitra column reports Qwen3.5-Mitra-9B, the sibling
107
- checkpoint of this model (same base and training recipe; on gold pools the
108
- two differ by at most ~2 points per task):
109
 
110
- | task | Qwen3.5-Mitra-9B (sibling)* | gemma-2-mitra-e | BGE-M3 |
111
  |---|---|---|---|
112
- | Multilingual parallels (sa/bo/zh) | **0.862** | 0.841 | 0.001 |
113
- | Multilingual asymmetric (sentence→window) | **0.789** | 0.581 | 0.000 |
114
- | English → classical | **0.900** | 0.879 | 0.068 |
115
- | Verse → commentary | **0.501** | 0.384 | 0.138 |
116
- | English question → passage | **0.522** | 0.473 | 0.020 |
117
-
118
- \* sibling checkpoint, not this exact model — see note above.
119
 
120
  **This checkpoint's own results** on gold pools (per-subtask pools without
121
  added distractors):
 
102
  ## Evaluation
103
 
104
  R@1 on the mitra retrieval benchmark, **hardened setting: 25,000 in-language
105
+ distractor passages added to every candidate pool** (this model's own
106
+ measurements; Tibetan given in native script):
 
 
107
 
108
+ | task | this model | gemma-2-mitra-e | BGE-M3 |
109
  |---|---|---|---|
110
+ | Multilingual parallels (sa/bo/zh) | **0.846** | 0.841 | 0.001 |
111
+ | Multilingual asymmetric (sentence→window) | **0.788** | 0.581 | 0.000 |
112
+ | English → classical | 0.876 | 0.879 | 0.068 |
113
+ | Verse → commentary | **0.483** | 0.384 | 0.138 |
114
+ | English question → passage | **0.523** | 0.473 | 0.020 |
 
 
115
 
116
  **This checkpoint's own results** on gold pools (per-subtask pools without
117
  added distractors):