GLM-5.2 Vocabulary Pruning Candidates (`glm_5.2_prune_candidates.json`)

Martial Terran, the LLM Vocabulary Reduction Specialist, has made a curated list [glm_5.2_prune_candidates.json] of Token IDs that are candidates for pruning or deactivation from the GLM-5.2 model's active inference weights, tokenizer configuration, and output projection layers.

The pruning configuration in glm_5.2_prune_candidates.json maps directly to rows that can be safely modified or removed within:

config.json
tokenizer_config.json
tokenizer.json
chat_template.jinja
Input Embeddings: model.embed_tokens.weight {Shape: torch.Size([154880, 6144])} in shard model-00001-of-00282.safetensors
LM Head Projection: lm_head.weight {Shape: torch.Size([154880, 6144])} in shard model-00001-of-00282.safetensors

Target Repository ID: zai-org/GLM-5.2

By safely deactivating or pruning these identified Token IDs from the active lm_head and embed_tokens matrices, you can reduce the parameter footprint and improve inference throughput.

Pruning Efficiency Gains

Pruning these 331 inactive tokens translates to a reduction of ~20.34 Billion MACs per 10,000 tokens of processed context.

Resource Reduction Report

Number of Dead Tokens Identified: 331
Vocabulary Reduction: 0.21% of the total vocabulary
Memory Saved (Raw Weights): 7.76 MB (at FP16/BF16 precision)
Total Parameters Removed: 4,067,328 parameters

Inference Efficiency (per 10,000 tokens)

MACs Saved: 20.34 Billion MACs
FLOPs Saved: 40.67 GFLOPs

Hardware-Aware Performance Profile

To evaluate the real-world performance impact of pruning these tokens, we look beyond raw FLOP counts. During LLM deployment, execution is split into two fundamentally different hardware regimes:

Prefill Phase (Prompt Processing): Compute-bound (GEMM). FLOP/MAC savings scale linearly with context length and batch size.
Decode Phase (Autoregressive Generation): Memory-bandwidth bound (GEMV). The bottleneck is streaming the weight matrices from GPU High Bandwidth Memory (HBM) to SRAM/Registers. FLOP savings are secondary to HBM read throughput savings.

1. Bandwidth Savings vs. FLOP Savings

During the Decode Phase, GPU execution is heavily bound by memory bandwidth. By removing the elements identified in glm_5.2_prune_candidates.json from the lm_head, we save 3.14 MB of HBM reads per token generation step (independent of batch size, as the weight matrix is streamed to SRAM once per step).

2. Latency Profile Generation

The table below translates these memory bandwidth savings into theoretical clock-time savings across NVIDIA A100, H100, and H200 accelerators based on raw hardware throughput limits.

GLM-5.2 Hardware-Aligned Pruning Report

================================================================================
      GLM-5.2 HARDWARE-ALIGNED PRUNING REPORT (d_model=6144)
================================================================================
--- 1. VOCABULARY ALIGNMENT ANALYSIS (CUDA Warp / Tensor Core Optimization) ---
Original Vocabulary Size:          154,880 (Multiple of 128)
Raw Dead Tokens Identified:        331 (0.214% of vocab)
...
--- 3. PREFILL PHASE PROFILE (Compute-Bound: GEMM) ---
Execution Target Context:          8,192 tokens (Batch Size: 8)
Computational MACs Saved:          103.0792 Billion MACs
Computational FLOPs Saved:         206.1584 GFLOPs

--- 4. DECODE PHASE PROFILE (Bandwidth-Bound: GEMV) ---
HBM Data Stream Saved per Step:    3072.00 KB / token step
Theoretical Hardware Latency Saved per Autoregressive Step:
  * NVIDIA A100 (SXM4)   | Bandwidth: 2.039 TB/s | Saved: 1542.78 ns
  * NVIDIA H100 (SXM5)   | Bandwidth: 3.350 TB/s | Saved: 939.02 ns
  * NVIDIA H200 (SXM)    | Bandwidth: 4.800 TB/s | Saved: 655.36 ns
================================================================================

================================================================================
          GLM-5.2 INFRASTRUCTURE FINANCIAL SAVINGS REPORT
================================================================================
Operational Parameters:
  * Electricity Rate:   $0.12 / kWh     |  Data Center PUE: 1.25
  * GPU Cloud Rate:     $7.00 / hr (H100) |  Daily Token Vol: 15,000M
  * Per 256 Tokens Pruned:
--------------------------------------------------------------------------------

--- 1. COST REDUCTIONS PER 1,000,000 INPUT TOKENS (GEMM Prefill) ---
  * Theoretical FLOPs Saved:         3,145,728,000,000 FLOPs
  * Active GPU Compute Time Saved:   7.9518 milliseconds
  * Pure Electrical Savings:         $0.00000015 / Million input tokens
  * Cloud Compute Rental Savings   : $0.000015 / Million input tokens

--- 2. COST REDUCTIONS PER 1,000,000 OUTPUT TOKENS (GEMV Decode) ---
  * HBM Memory Reads Saved:          3000000.00 MB
  * Active GPU Memory Streaming Time Saved: 1252.0310 milliseconds
  * Pure Electrical Savings:         $0.00002348 / Million output tokens
  * Cloud Compute Rental Savings   : $0.002435 / Million output tokens

--- 3. OPPORTUNISTIC KV CACHE RECOVERY ---
  * Static Parameter Memory Recovered: 6.00 MB
  * KV Cache Memory footprint/token:  80.00 KB
  * Extra Concurrent Cached Tokens Host-able: 77 tokens
    (Allows server farms to keep more prompt context cached in VRAM instead of reloading)

--- 4. SCALED SERVER FARM PROJECTIONS (At 15,000M tokens/day) ---
  * Combined GPU-Hours Reclaimed Daily: 1.0699 hours
  * Daily Infrastructure Cost Savings:  $7.49
  * Projected Annualized Cost Savings:  $2,735.38
================================================================================

In the context of an AI inference provider like Fireworks AI, the term **"Cloud Compute Rental Savings"** represents a combination of direct expense reductions, reclaimed capacity (opportunity cost), and improvements to their Cost of Goods Sold (COGS). 

Here is how a serverless inference provider like Fireworks AI experiences these "rental" or compute infrastructure savings in practice:

---

### 1. The Reality of GPU Infrastructure (Leasing vs. Owning)
Most serverless AI API providers do not build, own, or maintain physical data centers. Purchasing tens of thousands of H100 or H200 GPUs, cooling systems, and networking switches requires immense capital expenditure (CapEx). 

Instead, companies like Fireworks AI, Together AI, and DeepInfra typically **lease or rent their GPU fleets** from specialized "GPU cloud" providers (such as CoreWeave, Lambda Labs, Crusoe, or Oracle Cloud Infrastructure) on multi-year or annual contracts. 
* They pay a fixed hourly rate per active GPU (e.g., $2.00 to $3.50 per hour for a reserved H100, or up to $7.00 per hour for on-demand/spot instances). 
* Any optimization that reduces the GPU-seconds required to generate a token directly lowers the billing hours on their active cloud contracts.

---

### 2. How the Savings Materialize in Their Business Model

#### Scenario A: Direct Reduction in On-Demand/Spot Billing (Dynamic Scaling)
Serverless API traffic fluctuates heavily throughout the day. To handle peak hours, providers use auto-scaling systems that dynamically rent extra GPU instances (on-demand or spot instances) from cloud providers. 
* When traffic drops, they spin these instances down.
* If a model's decode phase is made more efficient, the workload is completed faster. The auto-scaler can spin down those expensive on-demand GPUs sooner, **directly reducing their hourly rental invoice** from the underlying GPU cloud.

#### Scenario B: Reclaimed Capacity & Capital Efficiency (The Reserved Fleet)
For their core traffic, providers lease a baseline number of GPUs on fixed 1-year or 3-year contracts. Since they pay a flat monthly lease fee for these GPUs whether they are idling or at 100% load, "rental savings" here represents **Opportunity Cost**.
* If pruning dead tokens makes the GLM-5.2 model 0.5% faster during the memory-bandwidth-bound decode phase, that fixed GPU fleet can now process **0.5% more concurrent requests** without any additional hardware.
* Instead of having to lease *more* GPUs to handle their growing user base, they "reclaim" existing capacity. They avoid having to sign a new lease contract, which represents a direct capital saving.

---

### 3. Impact on COGS (Cost of Goods Sold) and Gross Margins
For an API provider, the pricing model is usually structured as a dollar rate per million tokens (e.g., $1.40/M input, $4.40/M output). Their profit margin is determined by:

 $\text{Gross Margin} = \text{Price Charged to User} - \text{COGS (Compute Cost to Generate those Tokens)}$ 

Because the primary component of their COGS is the amortized hourly cost of the leased GPUs, reducing the GPU-milliseconds per token directly lowers their COGS. 
* By lowering their compute cost per token, they can either pocket the difference as **higher gross margin**, or **lower their API prices** to undercut competitors (like Together AI or DeepInfra) without sacrificing profitability.

GLM-5.2 Vocabulary Pruning Candidates (glm_5.2_prune_candidates.json)