---
license: other
license_name: shapleymcg-license-1.0
license_link: https://github.com/brandonmmusic-max/shapleymcg/blob/main/LICENSE
base_model:
  - brandonmusic/GLM-5.3-Flash-tr3-4bpw
  - drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45
pipeline_tag: text-generation
language:
  - en
  - zh
tags:
  - glm
  - glm-5.3-flash
  - exl3
  - tr3
  - shapleymcg
  - abliterated
  - vllm
---

# GLM-5.3-Flash EXL3 4bpw — Keys/Dealign o_proj transplant

An EXL3 derivative of **[brandonmusic/GLM-5.3-Flash-tr3-4bpw](https://huggingface.co/brandonmusic/GLM-5.3-Flash-tr3-4bpw)**,
using the **L15–43 + MTP L45** attention-output tensor selection shipped by
**[drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45)**.

**EXL3 expert weights, Keys/Dealign BF16 attention-output projections.** This is
not an NVFP4 checkpoint, a fresh quantization, or a runtime refusal-direction
projection. Layers **0–14 and 44** remain from the EXL3 parent.

> **Evaluation summary:** the fixed 50-question GSM8K sanity check remained in
> the same ballpark, but fidelity to the BF16 teacher decreased: KLD increased
> **2.77×** and realized-token perplexity increased **3.9%**. The donor's
> Refusal32 suite, run unmodified against this derivative, reproduced their
> result: **32/32 bypass, 0 refusals, 0 garbled, 0 empty** (greedy, thinking
> off; regex classifier, single run).

## Model overview

| Property | Description |
|---|---|
| Architecture | GLM-5.3-Flash; hybrid KDA/DSA attention with mHC |
| Hidden dimension | 4096 |
| Layers | 45 main layers plus native MTP layer 45; zero-based indices |
| Routed experts | 288, with top-8 routing |
| Quantization | Uniform-K4 EXL3/TR3 routed-expert quantization inherited from the parent |
| Edited weights | 30 BF16 `self_attn.o_proj.weight` tensors |
| Preserved weights | All weights outside the selected attention-output tensors |
| Checkpoint layout | 120 safetensors shards; approximately 175.64 GB of shard data |
| Runtime requirement | A compatible custom EXL3 runtime; not stock vLLM |
| Evaluation scope | Text quality comparison plus limited vision, tool, streaming and DFlash2 functional checks; video untested |

## Sources and provenance

| Component | Source | Contribution |
|---|---|---|
| EXL3 checkpoint body | [Brandon Music / ShapleyMCG](https://huggingface.co/brandonmusic/GLM-5.3-Flash-tr3-4bpw) | Quantized experts, non-selected weights, configuration, tokenizer, index and encoder closure |
| Tensor donor and selection recipe | [Keys / drowzeys](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45) | BF16 output projections for L15–43 and MTP L45 |
| Donor tensor origin | [Dealign](https://huggingface.co/dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4) | Edited attention-output tensors incorporated into Keys |
| Keys donor body | [RedHatAI GLM-5.3-Flash NVFP4](https://huggingface.co/RedHatAI/GLM-5.3-Flash-NVFP4) | Donor lineage only; its NVFP4 experts are **not** copied into this EXL3 model |
| Upstream model | [Z.ai GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) / [BF16](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16) | Original model and architecture |

Pinned source checkpoint revisions:

```text
EXL3 parent: aba59d2175e1ee2887ae0ae1300ba848b1deed84
Keys donor:  80b6d18d77e3020f2384597081d405f19893f101
```

Upstream method and implementation references:

- [Brandon Music's EXL3 repository](https://github.com/brandonmmusic-max/glm-5.3-flash-exl3-4bpw)
- [Keys' transplant repository](https://github.com/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45)
- [Keys METHOD.md at the pinned donor revision](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45/blob/80b6d18d77e3020f2384597081d405f19893f101/METHOD.md)

## Methodology

### Exact tensor transplant

The following BF16 tensors were copied byte-for-byte from the Keys donor:

```text
model.language_model.layers.{L}.self_attn.o_proj.weight
L = 15, 16, ..., 43, 45
```

This is **30 tensors**, not 31. Layer 44 is excluded.

| Weight group | Treatment |
|---|---|
| L15–43 attention output projections | Copy from Keys donor |
| MTP L45 attention output projection | Copy from Keys donor |
| L0–14 and L44 attention output projections | Preserve EXL3 parent |
| Experts, routers, QKV, embeddings, norms, vision and all other weights | Preserve EXL3 parent |
| Configuration, tokenizer and tensor index | Preserve EXL3 parent |

The donor's `METHOD.md` and `ABLIT_META.json` specify a **tensor transplant**, not
a rank-one projection. Its included refusal-direction `.pt` files are historical
experiments and were not applied. No training, direction estimation,
calibration, dequantization or requantization was performed.

The checkpoints use different expert packing formats, but the selected output
projections are already BF16 in both. KDA output projections have shape
`[4096, 8192]`; DSA and MTP output projections have shape `[4096, 16384]`.
Exactly **2,617,245,696 bytes** of tensor payload were transplanted.

### Why preserve the anchors?

The Keys recipe preserves early layers and excludes L44. Its authors identified
a large Dealign L44 change as a potential garbling risk in their NVFP4
experiments. This derivative follows the shipped selection rather than the
older L15–45 recipe that included L44.

For the pinned checkpoints used here, L44 is byte-identical between the donor
and EXL3 parent. Selected-tensor relative Frobenius deltas, measured in FP64,
range from **0.041501 to 0.187423**. Early layers remain from the EXL3 parent,
including those with small donor/parent differences.

### Integrity verification

- All **120 parent shards** and **11 donor shards** were hash-verified against
  their pinned Hugging Face content addresses.
- Every edited output shard was reread and verified.
- Every transplanted tensor matches its donor hash.
- All weight bytes outside the 30 selected tensor ranges match the EXL3 parent.
- Tensor index/header correspondence, configuration and EXL3 storage ABI were
  preserved and checked across **150,226 serialized tensors**, including
  quantization auxiliaries.
- The parent's encoder source-closure verifier,
  `reproducibility/r10/verify_bundle.py`, passed for **41 files**.
- **47 unit tests** passed for the build, launcher and evaluation tooling.

The transplant was performed without modifying the source checkpoints. A
reproduction must resolve tensor offsets from each checkpoint's own safetensors
headers: **do not reuse NVFP4 shard offsets for EXL3 files**. Verify the complete
output before making it available for inference.

Integrity verification establishes which bytes changed; the Refusal32 run below provides limited behavioral evidence, not a general guarantee of
unchanged model behavior.

## Evaluation results

The parent and derivative were evaluated using matching runtime settings and
hardware. These results were measured for this comparison, not inherited from
either source model card.

| Measurement | Original EXL3 | This derivative |
|---|---:|---:|
| Mean teacher KLD, 25 windows / 51,175 positions | 0.024754893 | 0.068454542 |
| Teacher top-1 agreement, 25 windows | 95.34% | 91.90% |
| Realized-token NLL, 25 windows | 1.064222869 | 1.102432383 |
| Mean teacher KLD, clean 17 windows / 34,799 positions | 0.025328698 | 0.073521263 |
| GSM8K first 50, zero-shot greedy | 48/50 | 49/50 |
| First 5 within that GSM8K subset | 5/5 | 5/5 |
| GSM8K API errors / truncated answers | 0 / 0 | 0 / 0 |

### Interpretation

No GSM8K question regressed in the fixed sample. The derivative answered one
additional question correctly, but this small difference does **not** establish
a general accuracy improvement.

Full-panel KLD increased **2.765×**, teacher top-1 agreement decreased **3.44
percentage points**, and realized-token perplexity increased **3.895%**. The
clean-17 subset shows the same direction of drift.

The results support retained capability on a limited sanity check, **not
unchanged fidelity, full-benchmark equivalence, or a safety claim**. KLD measures
changes in the output distribution; it is not itself a task-accuracy score.

### Fidelity protocol

- Teacher dataset: [GLM-5.3-Flash BF16 Teacher Logits](https://huggingface.co/datasets/brandonmusic/GLM-5.3-Flash-BF16-Teacher-Logits),
  revision `95f4fdd94bf29989db2e0d1054e4931f55edb6aa`.
- BF16 reference model revision: `a6c167b62691b2bac901344b65cb651a70f53e43`.
- All **25 sealed final windows**, each containing 2048 input tokens and all
  **2047 next-token prediction positions**. Teacher logits and token identities
  were verified. These windows were not used for fitting or tuning.
- Full-vocabulary prompt logits were captured before top-k selection. Input IDs
  and contiguous absolute row alignment were checked; row `r` predicts token
  `r+1`.
- The parent's published FP64 scoring kernel was reused to compute
  **KL(teacher || student)** in nats. The 24 padded columns were excluded,
  reducing stored vocabulary 154880 to real vocabulary 154856 before FP64
  log-softmax and accumulation.
- Clean-17 uses the parent's published calibration-overlap exclusions; that
  overlap audit was not independently repeated.

### GSM8K protocol

The evaluation used the official [OpenAI GSM8K test set](https://github.com/openai/grade-school-math/blob/3101c7d5072418e28b9008a6636bde82a006892c/grade_school_math/data/test.jsonl)
at revision `3101c7d5072418e28b9008a6636bde82a006892c`:

- Fixed **first 50 questions**, identical zero-shot prompts for both models.
- Thinking enabled; greedy decoding; seed 42; maximum 2048 output tokens.
- Prompts requested step-by-step reasoning and a final `####` numeric answer.
- Only explicit final numeric answers in `content` were graded. API failures
  and truncated responses counted as failures.

This is **not the full GSM8K benchmark**. The first-5 subtotal is part of the
same evaluation, not an exact reproduction of the upstream stochastic
five-question DFlash acceptance probe.

### Evaluation configuration

The comparison used two NVIDIA GB10 GPUs (SM121) with a compatible custom vLLM
EXL3 runtime based on [MiaAI-Lab's implementation](https://github.com/MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks)
at source revision `6599585dd5ab0b5f1f68e84914f48825727ad1b3`.

Both models used tensor parallelism 2, FP8 KV cache, eager execution, prefix
caching disabled, speculation disabled, no additional runtime abliteration,
text-only inference, context length 8192, one sequence and 2048-token prefill
chunks. The E2 EXL3 path was used (`EXL3_FAT_GROUPED=0`).

Each model/window was measured once in the paired panel evaluation. Cold-run
variance and formal statistical equivalence were not established. Because
speculation was disabled, these quality scores **do not evaluate the edited
MTP head**.

### Additional functional checks

Separate functional validation with native MTP k=2 passed:

- **12 benign API smoke checks**.
- Reasoning-output parsing and tool-call parsing.
- Retrieval from a **7463-token prompt**.

These checks exercised the edited MTP layer but were not controlled MTP
acceptance or quality benchmarks. They do not establish full-context or
throughput performance.

A subsequent shared-profile test ran the original and derivative sequentially
with **DFlash2 k=7, vision enabled, TP2, FP8 KV and CUDA graphs**. Both passed
**10/10 functional checks**:

| Check | Original EXL3 | This derivative |
|---|---|---|
| Arithmetic, JSON and Chinese answers | Pass | Pass |
| Reasoning parsing and tool calls | Pass | Pass |
| Vision: identify red/blue halves of a synthetic image | Pass | Pass |
| Retrieval from a 9,128-token prompt | Pass | Pass |
| Three concurrent requests | Pass | Pass |
| Streaming output | Pass | Pass |
| Active DFlash drafting and token acceptance | Pass | Pass |

The intended checkpoint was verified on both ranks, with the same API model
name and runtime settings for both variants. No OOM kills or container restarts
were observed during these checks. The profile had a 1,000,000-token configured
context ceiling, but **no full-1M-context request was tested**. Positive draft
and accepted-token counters establish that speculation was active, not a
controlled DFlash acceptance-rate or speed benchmark. The synthetic image test
is a vision smoke check, not a comprehensive multimodal quality evaluation.

## Upstream claims versus this model's results

| Upstream evidence | Scope |
|---|---|
| EXL3 model-card KLD 0.02455456425 | Published full-25 decoded-model evidence for the parent, not this derivative. The matched parent baseline measured here is 0.024754893. |
| EXL3 v75 FP8 KLD 0.02461059122 | Five runs of one complete 2047-position window, not five full 25-window panels. |
| EXL3 speed, capacity, DFlash acceptance and vision results | Hardware- and runtime-specific upstream results; not inherited by this derivative. |
| Keys “32/32 bypass, 0 refuse, 0 garble” | Donor authors' NVFP4 result. **Reproduced on this EXL3 derivative**: 32/32 bypass, 0 refuse, 0 garble with the donor's unmodified suite (single run, regex classifier, greedy, thinking off). |

Matching tensor edits across differently quantized expert bodies does not
establish identical refusal behavior.

## Usage and compatibility

Use a runtime that explicitly supports the parent's **EXL3/TR3/MCG format**.
The parent documents a custom SM120/B12X path, while
[MiaAI-Lab provides a separate SM121 implementation](https://github.com/MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks).
Follow the runtime's hardware-specific installation and launch instructions.
Stock vLLM compatibility is **not** claimed.

Important configuration considerations:

- **Do not apply another abliteration pass.** The weights are already edited.
  If the runtime exposes `ABLIT`, set **`ABLIT=0`**. An older L15–45 runtime
  recipe includes L44 and would apply a second, different modification.
- Keep the supplied EXL3 quantization configuration. Do not relabel the
  checkpoint `compressed-tensors` or `modelopt_fp4`, and do not use the donor's
  NVFP4-specific loading recipe for these weights.
- Native MTP is retained. External DFlash2 is a separate optional checkpoint
  with its own compatibility and licensing requirements; it was not used in
  the paired quality evaluation.
- Chat templates and reasoning parsers affect behavior. The donor's
  [thinking-off template](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45/blob/80b6d18d77e3020f2384597081d405f19893f101/chat_template.thinking-off.jinja)
  is a serving overlay, not part of the weight edit. Its authors warn that their
  stock template can ignore `enable_thinking=false`. Keep template choices
  explicit rather than assuming a flag alone disables reasoning.

## Limitations

- **Refusal evidence is a single regex-classified 32-prompt run** (donor's suite,
  unmodified); no human-rated or multi-run refusal evaluation.
- GSM8K results cover only a fixed 50-question subset, not general capability.
- Distributional fidelity measurably decreased after the transplant.
- No full-128K or full-1M context stress test, controlled throughput benchmark,
  comprehensive vision/video evaluation, MTP acceptance-quality benchmark or
  controlled DFlash2 performance benchmark was performed for this derivative.
- Retaining a weight component or configuration setting does not establish its
  functional quality in every runtime.
- Results may vary with hardware, kernels, KV-cache precision, templates,
  sampling configuration and speculative decoding.

## License and acknowledgements

This derivative retains the EXL3 parent's **ShapleyMCG License 1.0**:

- [Parent checkpoint license](https://huggingface.co/brandonmusic/GLM-5.3-Flash-tr3-4bpw/blob/aba59d2175e1ee2887ae0ae1300ba848b1deed84/LICENSE)
- [ShapleyMCG license](https://github.com/brandonmmusic-max/shapleymcg/blob/main/LICENSE)
- [Parent third-party notices](https://huggingface.co/brandonmusic/GLM-5.3-Flash-tr3-4bpw/blob/aba59d2175e1ee2887ae0ae1300ba848b1deed84/THIRD_PARTY_NOTICES.md)

The donor's MIT declaration does **not** relicense the combined EXL3 checkpoint
as MIT. Retain the applicable license files, third-party notices, encoder
closure and bundled third-party license texts when redistributing the model.
Review all applicable upstream terms before use or redistribution.

Credits:

- **Brandon Music / ShapleyMCG** — EXL3 checkpoint, encoder reproducibility
  closure, teacher-logit assets and fidelity evaluation harness.
- **Keys / drowzeys** — anchored tensor selection and documented transplant
  method; **Dealign** — edited attention-output tensors; **RedHatAI** — donor
  checkpoint body; **Z.ai** — original GLM model.
- **OrcaRouter** — the donor's format guide, not a checkpoint body copied here;
  **Blackfrost Research** — the donor's historical direction reference, not a
  projection applied to this derivative.
- **MiaAI-Lab**, **Local Inference Lab / B12X**, **vLLM**, **turboderp / ExLlamaV3**,
  and the contributors named in the retained third-party notices — runtime and
  quantization foundations.
- **malaiwah** — fidelity-suite reference documentation; **OpenAI** — GSM8K.

These acknowledgements identify provenance, not endorsement.

## Responsible use

Treat the model as potentially having reduced safeguards. Apply appropriate
application-level controls, access restrictions and human review. No safety
certification is claimed; refusal evidence is a single automated 32-prompt run
with a regex classifier, not a safety evaluation.

Preserve and comply with applicable upstream access and responsible-use terms,
including the donor's [Responsible Use Agreement](https://huggingface.co/drowzeys/keys-GLM-5.3-Flash-NVFP4-ablit-l15-43-mtp-l45/blob/80b6d18d77e3020f2384597081d405f19893f101/RESPONSIBLE_USE.md).