Upload folder using huggingface_hub

README.md

@@ -2,23 +2,36 @@
 language: en
 tags:
   - tiny
-  - reasoning
-  - alibi
-  - gqa
+  - pct-v3
+  - rpw
+  - gpp
+  - vcr
 ---
 
-# Lumia Tiny (V2)
+# Lumia Tiny (PCT-V3)
 
-Custom PyTorch reasoning SFT model. ~1.1M params.
+Custom PyTorch LM ~969K params. Arsitektur dari first principles.
 
 ## Architecture
-- ALiBi + GQA (8 heads, 4 KV heads)
-- RMSNorm + SiLU gating
-- Tied embeddings, head_dim=16
-- Vocab: 1757 (no filler)
+- **RPW** — Relative Positional Warp (learned Fourier additive bias)
+- **GQA** — 8 query / 4 KV heads
+- **GPP** — Gated Principal Projection (96-dim bottleneck)
+- **VCR** — Variance-Controlled Residual (R² gating)
+- **ECI** — Entropy-Calibrated Initialization
+- Vocab: 4096, RMSNorm, tied embeddings, 6 layers
+
+## Files
+| File | Description |
+|---|---|
+| `model_tiny.py` | Arsitektur PCT-V3 (RPW, GQA, GPP, VCR, TinyModel) |
+| `train_tiny.py` | Training loop (IterableDataset, cosine LR, checkpoint) |
+| `train_tiny.yaml` | Konfigurasi training (LR 3e-4, batch 8, GA 4, 50k steps) |
+| `config.json` | HF config auto-map untuk TinyModel |
+| `tokenizer.json` | BPE tokenizer 4096 vocab |
+| `gen_tokenizer.py` | Generator tokenizer |
 
 ## Usage
 ```python
-from scripts.model_tiny import TinyModel
-model = TinyModel()  # V2 defaults
+from model_tiny import create_model
+model = create_model()  # PCT-V3 defaults
 ```