Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,92 @@
+---
+language: en
+license: mit
+tags:
+  - pronunciation-assessment
+  - phoneme-scoring
+  - wavlm
+  - speech
+  - children-speech
+base_model: microsoft/wavlm-large
+pipeline_tag: audio-classification
+---
+
+# WavLM Phoneme Scorer
+
+Fine-tuned **WavLM-Large** backbone + MLP scoring head for phoneme-level English pronunciation assessment, trained on 11K children's speech samples.
+
+Given a reference text and an audio recording, this model identifies which phonemes are pronounced correctly and which have errors.
+
+## Architecture
+
+```
+Audio ──→ CTC model (alignment) ──→ Frame-level phoneme segments
+  │
+  └──→ WavLM-Large (fine-tuned top 6 layers) ──→ Hidden states per segment
+                                                        │
+                                                  + phone embedding (32d)
+                                                  + GOP score (1d)
+                                                  + n_frames (1d)
+                                                        │
+                                                  MLP (1058 → 512 → 512 → 256)
+                                                        │
+                                                  ├── score_head → phoneme score (0-100)
+                                                  └── pherr_head → error probability (0-1)
+```
+
+- **Backbone**: WavLM-Large with top 6 transformer layers fine-tuned (76.6M trainable / 316.4M total)
+- **Alignment**: `facebook/wav2vec2-xlsr-53-espeak-cv-ft` (frozen, CTC-based Viterbi forced alignment)
+- **Scoring head**: MLP with BatchNorm, GELU, Dropout, multi-task (score regression + error classification)
+
+## Performance
+
+Evaluated on test set (8062 phonemes from 1727 audio files):
+
+| Metric | GOP Baseline (v1.0) | This Model |
+|--------|-------------------|------------|
+| Phoneme Error AUC-ROC | 0.738 | **0.870** |
+| Phoneme Error F1 | 0.476 | **0.595** |
+| Phoneme Error Precision | 0.379 | **0.592** |
+| Phone Score Pearson | 0.372 | **0.645** |
+| Phone Score MAE | 27.44 | **16.47** |
+
+## Usage
+
+```python
+from pipeline_v2 import PronunciationAssessorV2
+
+assessor = PronunciationAssessorV2(checkpoint_path="wavlm_finetuned.pt")
+result = assessor.assess("audio.mp3", "Hello, Peter.")
+
+# result["overall_score"]  → 82.3
+# result["words"][0]["phonemes"][0]["error"]  → False
+# result["words"][0]["phonemes"][0]["pherr_prob"]  → 0.05
+```
+
+CLI:
+```bash
+python pipeline_v2.py --audio audio.mp3 --text "Hello, Peter."
+```
+
+See full pipeline code at: [github.com/Jianshu-She/Voice-correction](https://github.com/Jianshu-She/Voice-correction)
+
+## Training Data
+
+- 11,601 audio recordings of English learners (children)
+- 53,926 phonemes with professional human evaluation labels
+- Labels include per-phoneme scores (0-100) and error flags
+
+## Training Details
+
+- Fine-tuned top 6 of 24 WavLM transformer layers
+- Differential learning rate: backbone 1e-5, head 5e-4
+- AdamW optimizer, cosine annealing, early stopping (patience=8)
+- Gradient accumulation (4 steps), gradient clipping (max_norm=1.0)
+- Batch size 64, trained for 24 epochs (early stopped)
+- Multi-task loss: MSE/100 (score) + BCE with pos_weight=5.2 (error)
+
+## Files
+
+- `wavlm_finetuned.pt` — Full checkpoint (backbone + head state dict, 1.2GB)
+- `pipeline_v2.py` — Inference pipeline
+- `finetune_wavlm.py` — Training script