Create GLM-5.2 Vocabulary Efficiency Improvements.md

GLM-5.2 Vocabulary Efficiency Improvements.md

@@ -0,0 +1,73 @@
+GLM-5.2 Vocabulary Efficiency Improvements.md
+
+# GLM-5.2 Vocabulary Pruning Candidates (`glm_5.2_prune_candidates.json`)
+
+Martial Terran, the LLM Vocabulary Reduction Specialist, has made a curated list [glm_5.2_prune_candidates.json] of Token IDs that are candidates for pruning or deactivation from the GLM-5.2 model's active inference weights, tokenizer configuration, and output projection layers.
+
+The pruning configuration in `glm_5.2_prune_candidates.json` maps directly to rows that can be safely modified or removed within:
+* `config.json`
+* `tokenizer_config.json`
+* `tokenizer.json`
+* `chat_template.jinja`
+* **Input Embeddings:** `model.embed_tokens.weight` `{Shape: torch.Size([154880, 6144])}` in shard `model-00001-of-00282.safetensors`
+* **LM Head Projection:** `lm_head.weight` `{Shape: torch.Size([154880, 6144])}` in shard `model-00001-of-00282.safetensors`
+
+**Target Repository ID:** `zai-org/GLM-5.2`
+
+By safely deactivating or pruning these identified Token IDs from the active `lm_head` and `embed_tokens` matrices, you can reduce the parameter footprint and improve inference throughput.
+
+---
+
+## Pruning Efficiency Gains
+
+Pruning these 331 inactive tokens translates to a reduction of **~20.34 Billion MACs** per 10,000 tokens of processed context.
+
+### Resource Reduction Report
+* **Number of Dead Tokens Identified:** 331
+* **Vocabulary Reduction:** 0.21% of the total vocabulary
+* **Memory Saved (Raw Weights):** 7.76 MB (at FP16/BF16 precision)
+* **Total Parameters Removed:** 4,067,328 parameters
+
+### Inference Efficiency (per 10,000 tokens)
+* **MACs Saved:** 20.34 Billion MACs
+* **FLOPs Saved:** 40.67 GFLOPs
+
+---
+
+## Hardware-Aware Performance Profile
+
+To evaluate the real-world performance impact of pruning these tokens, we look beyond raw FLOP counts. During LLM deployment, execution is split into two fundamentally different hardware regimes:
+
+1. **Prefill Phase (Prompt Processing):** Compute-bound (GEMM). FLOP/MAC savings scale linearly with context length and batch size.
+2. **Decode Phase (Autoregressive Generation):** Memory-bandwidth bound (GEMV). The bottleneck is streaming the weight matrices from GPU High Bandwidth Memory (HBM) to SRAM/Registers. FLOP savings are secondary to **HBM read throughput savings**.
+
+### 1. Bandwidth Savings vs. FLOP Savings
+During the **Decode Phase**, GPU execution is heavily bound by memory bandwidth. By removing the elements identified in `glm_5.2_prune_candidates.json` from the `lm_head`, we save **3.14 MB of HBM reads per token generation step** (independent of batch size, as the weight matrix is streamed to SRAM once per step).
+
+### 2. Latency Profile Generation
+The table below translates these memory bandwidth savings into theoretical clock-time savings across NVIDIA A100, H100, and H200 accelerators based on raw hardware throughput limits.
+
+---
+
+## GLM-5.2 Hardware-Aligned Pruning Report
+
+```text
+================================================================================
+      GLM-5.2 HARDWARE-ALIGNED PRUNING REPORT (d_model=6144)
+================================================================================
+--- 1. VOCABULARY ALIGNMENT ANALYSIS (CUDA Warp / Tensor Core Optimization) ---
+Original Vocabulary Size:          154,880 (Multiple of 128)
+Raw Dead Tokens Identified:        331 (0.214% of vocab)
+...
+--- 3. PREFILL PHASE PROFILE (Compute-Bound: GEMM) ---
+Execution Target Context:          8,192 tokens (Batch Size: 8)
+Computational MACs Saved:          103.0792 Billion MACs
+Computational FLOPs Saved:         206.1584 GFLOPs
+
+--- 4. DECODE PHASE PROFILE (Bandwidth-Bound: GEMV) ---
+HBM Data Stream Saved per Step:    3072.00 KB / token step
+Theoretical Hardware Latency Saved per Autoregressive Step:
+  * NVIDIA A100 (SXM4)   | Bandwidth: 2.039 TB/s | Saved: 1542.78 ns
+  * NVIDIA H100 (SXM5)   | Bandwidth: 3.350 TB/s | Saved: 939.02 ns
+  * NVIDIA H200 (SXM)    | Bandwidth: 4.800 TB/s | Saved: 655.36 ns
+================================================================================