---
license: apache-2.0
pipeline_tag: token-classification
tags:
  - ner
  - gliner
  - data-use
---

# gliner-probe-smoke

Fine-tune of `urchade/gliner_large-v2.1` for data-use mention extraction
(dataset / survey / census / registry mentions in economics research papers).

## Labels
- `NAMED_DATA` — a proper name, title, or acronym of a specific data source
- `DESCRIPTIVE_DATA` — a source described in words but not named
- `VAGUE_DATA` — generic data wording with no identifiable source

## Training
- base model: `urchade/gliner_large-v2.1`
- dataset: `rafmacalaba/usage-sensitivity-probe` (gliner config)
- corpus: `all`
- epochs: 1
- learning rate: 5e-06
- batch size: 16
- precision: bf16

## Evaluation (holdout)

| thr | tp | fp | fn | precision | recall | f0.5 | f1 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 0.10 | 36 | 865 | 4 | 0.0400 | 0.9000 | 0.0494 | 0.0765 |
| 0.20 | 33 | 445 | 7 | 0.0690 | 0.8250 | 0.0845 | 0.1274 |
| 0.30 | 29 | 243 | 11 | 0.1066 | 0.7250 | 0.1285 | 0.1859 |
| 0.40 | 22 | 142 | 18 | 0.1341 | 0.5500 | 0.1580 | 0.2157 |
| 0.50 | 16 | 75 | 24 | 0.1758 | 0.4000 | 0.1980 | 0.2443 |
| 0.60 | 6 | 35 | 34 | 0.1463 | 0.1500 | 0.1471 | 0.1481 |
| 0.70 | 3 | 18 | 37 | 0.1429 | 0.0750 | 0.1210 | 0.0984 |

**Best F0.5**: 0.1980 (thr=0.5)
**Best F1**: 0.2443 (thr=0.5)

## Evaluation breakdown (holdout)

| group | examples | spans | thr | precision | recall | f0.5 | f1 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| overall | 64 | 40 | 0.50 | 0.1758 | 0.4000 | 0.1980 | 0.2443 |
| prwp | 14 | 12 | 0.50 | 0.1212 | 0.3333 | 0.1389 | 0.1778 |
| fcv | 50 | 28 | 0.50 | 0.2069 | 0.4286 | 0.2308 | 0.2791 |
| general_prwp | 14 | 12 | 0.50 | 0.1212 | 0.3333 | 0.1389 | 0.1778 |
| fcv_pads_east_africa | 27 | 11 | 0.50 | 0.1481 | 0.3636 | 0.1681 | 0.2105 |
| jdc_operational | 6 | 5 | 0.70 | 1.0000 | 0.2000 | 0.5556 | 0.3333 |
| refugee_pads | 5 | 3 | 0.60 | 1.0000 | 0.3333 | 0.7143 | 0.5000 |
| reliefweb | 12 | 9 | 0.50 | 0.3077 | 0.4444 | 0.3279 | 0.3636 |

### Per-label (overall)

| label | examples | spans | thr | precision | recall | f0.5 | f1 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| NAMED_DATA | 64 | 9 | 0.60 | 0.1200 | 0.3333 | 0.1376 | 0.1765 |
| DESCRIPTIVE_DATA | 64 | 10 | 0.30 | 0.0149 | 0.1000 | 0.0180 | 0.0260 |
| VAGUE_DATA | 64 | 21 | 0.50 | 0.1154 | 0.1429 | 0.1200 | 0.1277 |

<details>
<summary>Origin breakdown (per-origin metrics)</summary>

| origin | examples | spans | thr | precision | recall | f0.5 | f1 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| fcv_pads_east_africa | 27 | 11 | 0.50 | 0.1481 | 0.3636 | 0.1681 | 0.2105 |
| general_prwp | 14 | 12 | 0.50 | 0.1212 | 0.3333 | 0.1389 | 0.1778 |
| jdc_operational | 6 | 5 | 0.70 | 1.0000 | 0.2000 | 0.5556 | 0.3333 |
| refugee_pads | 5 | 3 | 0.60 | 1.0000 | 0.3333 | 0.7143 | 0.5000 |
| reliefweb | 12 | 9 | 0.50 | 0.3077 | 0.4444 | 0.3279 | 0.3636 |

</details>

<details>
<summary>Per-label details</summary>

| label | examples | spans | thr | precision | recall | f0.5 | f1 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| NAMED_DATA | 64 | 9 | 0.60 | 0.1200 | 0.3333 | 0.1376 | 0.1765 |
| DESCRIPTIVE_DATA | 64 | 10 | 0.30 | 0.0149 | 0.1000 | 0.0180 | 0.0260 |
| VAGUE_DATA | 64 | 21 | 0.50 | 0.1154 | 0.1429 | 0.1200 | 0.1277 |

</details>