unknown commited on
Commit
75c1b11
·
1 Parent(s): 28b352d

Create comprehensive model card README.md with evaluation metrics and usage guide

Browse files
Files changed (1) hide show
  1. README.md +158 -0
README.md ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: timm
3
+ tags:
4
+ - vision
5
+ - image-classification
6
+ - facial-expression-recognition
7
+ - vit
8
+ - vision-transformer
9
+ - fer
10
+ - ferplus
11
+ datasets:
12
+ - ferplus
13
+ metrics:
14
+ - accuracy
15
+ - f1
16
+ - precision
17
+ - recall
18
+ pipeline_tag: image-classification
19
+ ---
20
+
21
+ # ViT-Small for Facial Expression Recognition (FER++)
22
+
23
+ This repository contains a **Vision Transformer (ViT-Small)** model fine-tuned for **Facial Expression Recognition (FER)**.
24
+
25
+ The model is initialized from the pretrained checkpoint `vit_small_patch16_224.augreg_in1k` from the `timm` library and fine-tuned on a 7-class emotion classification dataset (aligned with FER++ taxonomy).
26
+
27
+ ## Model Details
28
+
29
+ - **Model Architecture:** Vision Transformer (ViT-Small)
30
+ - **Pretrained Base:** `vit_small_patch16_224.augreg_in1k`
31
+ - **Number of Parameters:** ~22M
32
+ - **Task:** 7-class Facial Expression Classification
33
+ - **Classes:**
34
+ - `0`: Angry
35
+ - `1`: Disgust
36
+ - `2`: Fear
37
+ - `3`: Happy
38
+ - `4`: Neutral
39
+ - `5`: Sad
40
+ - `6`: Surprise
41
+
42
+ ## Training Parameters & Hyperparameters
43
+
44
+ The model was fine-tuned using PyTorch and the `timm` library with the following setup:
45
+
46
+ | Hyperparameter | Value |
47
+ | :--- | :--- |
48
+ | **Epochs** | 50 |
49
+ | **Batch Size** | 128 |
50
+ | **Base Learning Rate** | 5e-4 |
51
+ | **Weight Decay** | 0.01 |
52
+ | **Optimizer** | AdamW |
53
+ | **Scheduler** | Cosine Annealing (with 5-epoch warmup starting at 1e-6, min LR 1e-5) |
54
+ | **Image Resolution** | 224 x 224 (Bicubic Interpolation) |
55
+ | **Stochastic Depth (Drop Path)** | 0.1 |
56
+ | **Precision** | Mixed Precision (AMP) |
57
+ | **Data Augmentations** | RandAugment, Random Erasing (p=0.1) |
58
+ | **Loss Function** | Weighted Cross-Entropy Loss |
59
+ | **Class Weights** | `[1.560, 3.737, 2.242, 0.541, 0.527, 0.970, 1.149]` (mapping to classes 0 to 6) |
60
+
61
+ ---
62
+
63
+ ## Evaluation Results
64
+
65
+ ### Validation Set Performance
66
+ - **Accuracy:** 83.83%
67
+ - **Precision (Macro):** 0.7892
68
+ - **Recall (Macro):** 0.7767
69
+ - **F1-Score (Macro):** 0.7817
70
+ - **F1-Score (Weighted):** 0.8378
71
+
72
+ #### Per-Class Validation Metrics:
73
+ | Class ID | Class Name | Precision | Recall | F1-Score | Support |
74
+ | :---: | :--- | :---: | :---: | :---: | :---: |
75
+ | 3 | Happy | 0.9247 | 0.9345 | 0.9295 | 473 |
76
+ | 6 | Surprise | 0.8636 | 0.7661 | 0.8120 | 124 |
77
+ | 4 | Neutral | 0.7943 | 0.8145 | 0.8043 | 275 |
78
+ | 0 | Angry | 0.7412 | 0.8400 | 0.7875 | 75 |
79
+ | 5 | Sad | 0.7758 | 0.7711 | 0.7734 | 166 |
80
+ | 2 | Fear | 0.8000 | 0.7273 | 0.7619 | 33 |
81
+ | 1 | Disgust | 0.6250 | 0.5833 | 0.6034 | 60 |
82
+
83
+ ---
84
+
85
+ ### Test Set Performance
86
+ - **Accuracy:** 83.68%
87
+ - **Precision (Macro):** 0.7800
88
+ - **Recall (Macro):** 0.7413
89
+ - **F1-Score (Macro):** 0.7565
90
+ - **F1-Score (Weighted):** 0.8348
91
+
92
+ #### Per-Class Test Metrics:
93
+ | Class ID | Class Name | Precision | Recall | F1-Score | Support |
94
+ | :---: | :--- | :---: | :---: | :---: | :---: |
95
+ | 3 | Happy | 0.9187 | 0.9378 | 0.9281 | 482 |
96
+ | 6 | Surprise | 0.8793 | 0.7969 | 0.8361 | 128 |
97
+ | 0 | Angry | 0.8158 | 0.8493 | 0.8322 | 73 |
98
+ | 5 | Sad | 0.8688 | 0.7277 | 0.7920 | 191 |
99
+ | 4 | Neutral | 0.7240 | 0.8707 | 0.7906 | 232 |
100
+ | 2 | Fear | 0.6471 | 0.5000 | 0.5641 | 22 |
101
+ | 1 | Disgust | 0.6066 | 0.5068 | 0.5522 | 73 |
102
+
103
+ ---
104
+
105
+ ## How to Use
106
+
107
+ Here is how you can load the model and run inference on an image using PyTorch and `timm`:
108
+
109
+ ```python
110
+ import torch
111
+ import torch.nn as nn
112
+ from torchvision import transforms
113
+ from PIL import Image
114
+ from timm import create_model
115
+
116
+ # Define class mapping
117
+ class_mapping = {
118
+ 0: "Angry",
119
+ 1: "Disgust",
120
+ 2: "Fear",
121
+ 3: "Happy",
122
+ 4: "Neutral",
123
+ 5: "Sad",
124
+ 6: "Surprise"
125
+ }
126
+
127
+ # 1. Define image preprocessing matching validation/test set config
128
+ transform = transforms.Compose([
129
+ transforms.Resize((224, 224), interpolation=transforms.InterpolationMode.BICUBIC),
130
+ transforms.ToTensor(),
131
+ transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
132
+ ])
133
+
134
+ # 2. Instantiate and build the model structure
135
+ model = create_model('vit_small_patch16_224.augreg_in1k', pretrained=False, num_classes=7)
136
+
137
+ # 3. Load weight state dictionary
138
+ state_dict = torch.load('vitsmall2.pth', map_location='cpu')
139
+
140
+ # Clean state_dict if it was saved using DataParallel (remove 'module.' prefix)
141
+ if list(state_dict.keys())[0].startswith('module.'):
142
+ state_dict = {k.replace('module.', ''): v for k, v in state_dict.items()}
143
+
144
+ model.load_state_dict(state_dict)
145
+ model.eval()
146
+
147
+ # 4. Perform Inference
148
+ image_path = "path_to_facial_image.jpg"
149
+ image = Image.open(image_path).convert('RGB')
150
+ input_tensor = transform(image).unsqueeze(0) # Add batch dimension
151
+
152
+ with torch.no_grad():
153
+ outputs = model(input_tensor)
154
+ probabilities = torch.softmax(outputs, dim=1)[0]
155
+ predicted_class_id = torch.argmax(probabilities).item()
156
+
157
+ print(f"Predicted emotion: {class_mapping[predicted_class_id]} ({probabilities[predicted_class_id].item():.2%})")
158
+ ```