headless-start/parameter-efficient-dfer

Parameter-Efficient Adaptation of Facial Expression Recognition Models for Driver Monitoring

Trained weights for the MSc thesis of Ayush Tiwari, BTU Cottbus-Senftenberg, 2026.

A ViT-B/16 (vit_base_patch16_224.augreg_in21k_ft_in1k) is first fine-tuned on a six-class subset of FER+ (Stage 1). Five adaptation strategies then transfer it to the KMU-FED in-vehicle expression corpus under ten-fold cross-validation (Stage 2): full fine-tuning, a linear probe, SSF, LoRA and AdaptFormer.

This repository holds every checkpoint the reported results come from: the Stage-1 backbone and the ten fold checkpoints of each strategy's selected configuration, 51 checkpoints in total, together with the per-fold metrics, per-frame predictions and selection records. No image data is included.

Classes, in output order: angry, disgust, fear, happy, sad, surprise.

Results

Stage 1: FER+ (six classes)

Learning rate selected from {1e-5, 3e-5, 1e-4} on the official validation split; the test split was evaluated once, for the selected checkpoint.

Learning rate	Epoch kept	Val. accuracy	Test accuracy	Test UAR	Test weighted F1
1e-5	28	94.36	93.21	81.40	93.13

Stage 2: KMU-FED, ten-fold cross-validation

Mean over the ten held-out folds (110 frames each); ± is the standard deviation across folds, reported as spread only.

Strategy	Selected configuration	Trainable parameters	Accuracy (%)	UAR (%)	Weighted F1 (%)
LoRA	lr 1e-3, rank 4	152,070 (0.177%)	99.00 ± 2.59	98.56	98.83
AdaptFormer	lr 1e-3, bottleneck 64 (reduction factor 12)	1,194,246 (1.373%)	98.45 ± 2.43	98.21	98.40
SSF	lr 3e-3	210,438 (0.245%)	97.91 ± 2.88	97.48	97.75
Full fine-tuning	lr 3e-5	85,803,270 (100%)	97.55 ± 4.11	96.87	97.25
Linear probe	lr 3e-3	4,614 (0.005%)	93.27 ± 6.83	92.76	92.42

How to read these numbers

• Folds are contiguous per-class blocks of frames, so the same drivers appear in training and test. The results describe familiar drivers, not unseen ones.
• Each fold keeps the epoch with the highest accuracy on its own held-out block, and each strategy's configuration was chosen by mean accuracy over the same ten folds. Both choices make the estimates optimistic. There is no separate validation split at Stage 2.
• One seed (42) was used throughout.

Repository layout

stage1/
  best.ckpt              full ViT-B/16 fine-tuned on FER+ (sha256 3094f103c17bd135...)
  results.json           validation history and test metrics
  study.json             the three learning-rate candidates and the selection rule
stage2/<strategy>/
  selection.json         every candidate's ten-fold mean accuracy and the selected configuration
  search_summary.json    summary of the strategy's search
  fold_00 ... fold_09/
    best.ckpt            retained checkpoint for that fold
    results.json         metrics, per-class recall, confusion matrix, training history
    predictions.json     label, prediction and probabilities for every held-out frame
load_weights.py          loads any checkpoint into the model of the code release

<strategy> is one of full_ft, linear_probe, ssf, lora, adaptformer. The Stage-1 and full fine-tuning checkpoints contain the whole network (343 MB). The linear probe, SSF, LoRA and AdaptFormer checkpoints contain only the trained parameters and the classification head (0.02 to 4.8 MB), which are applied on top of the Stage-1 encoder.

Usage

The checkpoints are PyTorch files with the weights under the model key. load_weights.py builds the matching model with methods.py from the thesis code release, so copy it next to that code:

from load_weights import load_stage2

model = load_stage2("lora", fold=0)   # downloads stage1/best.ckpt as well

Inputs are face crops converted to three-channel grayscale, resized to 224 x 224 with bicubic interpolation and normalised with mean and standard deviation 0.5 (build_transform(False, kmufed=True) in data.py). KMU-FED frames were cropped offline with MTCNN before training.

The loader was checked against the saved predictions: for every strategy and every fold it reproduces the stored prediction for all 110 held-out frames.

Checkpoints are pickled PyTorch files and need torch.load(..., weights_only=False); load them only from this repository.

Training summary

	Stage 1	Stage 2
Data	FER+ six-class subset: 17,651 train / 2,004 val / 2,047 test	KMU-FED, 1,106 frames; 996 train / 110 test per fold
Labels	official FER+ cross-entropy targets (train), majority vote (evaluation)	hard labels
Epochs	120	60 per fold
Optimiser	AdamW, weight decay 0.05, 2 warmup epochs then cosine to 1e-7	same
Batch, precision	32, mixed precision	same
Augmentation	grayscale flip, rotation ±15°, translation, scale 0.90-1.10, shear, blur, erasing	same, rotation ±30° and scale 0.75-1.15
Search	learning rate	learning rate, then rank (LoRA) or bottleneck width (AdaptFormer) at the selected rate

Data and licence

The weights are released for non-commercial research under CC BY-NC 4.0. They were trained on FER+ (FER2013 images with the Microsoft FER+ labels) and on KMU-FED, which is available from its authors on request; the terms of those datasets and of the pretrained ViT checkpoint continue to apply. The Stage-2 models were trained and evaluated on a small number of drivers and are not suitable for any decision about a real person.

Citation

@mastersthesis{tiwari2026peft_dfer,
  author = {Ayush Tiwari},
  title  = {Parameter-Efficient Adaptation of Facial Expression Recognition Models for Driver Monitoring},
  school = {Brandenburgische Technische Universit{\"a}t Cottbus-Senftenberg},
  year   = {2026}
}