bert-base-uncased-just-transition-augmented-v1.1

This model is a fine-tuned version of bert-base-uncased designed to classify parliamentary speech according to whether it contains references to Just Transition (JT) themes.

The model was developed as part of the research presented in:

Albanese, F. (2026). Uneven transitions: Measuring parliamentary engagement with just transition across European political economies. Energy Research & Social Science, 137, 104771. https://doi.org/10.1016/j.erss.2026.104771

The classifier was used to construct the Just Transition Index (JTI), a cross-national measure of parliamentary engagement with Just Transition themes across 16 European countries.

The model achieves the following results on the evaluation set:

Loss: 0.0377
Accuracy: 0.9875
F1: 0.9875

Model description

The model identifies parliamentary statements related to Just Transition discourse.

Intended uses.

This model is suitable for:

parliamentary speech analysis,
political agenda-setting research,
comparative political economy research,
climate politics research,
large-scale text classification pipelines,
exploratory analysis of Just Transition discourse.

Potential applications include:

constructing longitudinal salience indices,
comparing political parties or countries,
tracking discourse over time,
identifying JT-related debate segments for qualitative analysis.

Limitations

The model detects the presence of Just Transition discourse, not its normative orientation.

As a result:

supportive and oppositional references may both be classified as JT-related,
the model does not perform stance detection or sentiment analysis,
country-specific political framing may affect interpretation,
outputs should be interpreted as indicators of engagement/salience, not policy support.

The model was trained primarily on parliamentary discourse and may generalize less effectively to substantially different text domains.

Training procedure

Training and evaluation data

The model was trained on a relatively small but iteratively refined dataset of parliamentary-style text segments.

Training data included:

manually reviewed parliamentary excerpts,
synthetic examples generated for data augmentation,
active-learning-assisted corrections.

The dataset was designed to capture variation in how Just Transition themes appear in political discourse across institutional and national contexts.

Recommended classification threshold

Although the classifier outputs probabilities between 0 and 1, this project recommends using a high classification threshold of 0.96 for downstream inference and index construction.

This recommendation reflects the characteristics of the training setup:

the labelled dataset is relatively small,
label smoothing was not used during fine-tuning,
transformer classifiers under these conditions may produce overconfident probability estimates.

Using a higher threshold substantially reduces false positives and produces a more conservative and stable measure of parliamentary engagement with Just Transition themes. Users interested in recall-oriented applications may experiment with lower thresholds, but threshold sensitivity should be evaluated carefully depending on the research objective.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 16
seed: 1234
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 10
num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.6777	1.0	20	0.4837	0.8375	0.8331
0.4083	2.0	40	0.1511	1.0	1.0
0.2321	3.0	60	0.0745	0.975	0.9750
0.0534	4.0	80	0.1344	0.9625	0.9624
0.0042	5.0	100	0.0180	0.9875	0.9875
0.0022	6.0	120	0.0453	0.9875	0.9875
0.0024	7.0	140	0.0432	0.9875	0.9875
0.0014	8.0	160	0.0377	0.9875	0.9875

Citation

If you use this model in academic work, please cite:

@article{
  title = {Uneven transitions: Measuring parliamentary engagement with just transition across European political economies},
  journal = {Energy Research & Social Science},
  volume = {137},
  year = {2026},
  doi = {https://doi.org/10.1016/j.erss.2026.104771},
  author = {Francesco Albanese}
}

Data availability

The processed dataset and analysis code supporting this study/model are available in the Radboud Data Repository at: https://doi.org/10.34973/mhvk-e634.