oceanusm/ests-gptoss-arz-k26

ESTS GPT-OSS English-to-Egyptian-Arabic Specialist (avg-k=26)

This physically pruned, recovery-tuned, and MXFP4-packed checkpoint is specialized for English-to-Egyptian-Arabic translation. It is a CONTRASTIVE Egyptian Arabic ESTS checkpoint for the WMT 2026 Model Compression shared task.

Property	Value
Base model	openai/gpt-oss-20b
Average experts removed	26 per MoE layer
Experts retained	6 per layer on average; range 4-9
Parameter count	5.380B
Packed checkpoint size	Approximately 5.15 GiB
Recovery data	Approximately 10,000 English-to-Egyptian-Arabic examples

Quick start

git clone https://github.com/oceanusm/wmt26-ests-model-compression.git
cd wmt26-ests-model-compression

SUB=submissions/ests-gptoss-arz-k26
MODEL_CACHE="$HOME/.cache/ests-models" \
MODELZIP_SOURCE="$PWD" \
bash "$SUB/setup.sh"

printf '%s\n' 'The meeting begins at nine tomorrow morning.' > /tmp/ests-input.txt
CUDA_VISIBLE_DEVICES=0 bash "$SUB/run.sh" \
  --lang-pair eng-ara_EG \
  --batch-size 1 \
  --input /tmp/ests-input.txt \
  --output /tmp/ests-output.txt
cat /tmp/ests-output.txt

This checkpoint requires the custom variable-capacity MoE/MXFP4 implementation provided by the linked runner. It is not a drop-in stock Transformers model.

Method

Experts were ranked within each layer by accumulated routing mass, and language-specific layerwise allocation determined how many experts to retain. Router and expert tensors were physically sliced and compactly remapped, the pruned model was recovery-tuned, and the routed expert weights were packed to MXFP4.

Limitations

This checkpoint is intended for English-to-Egyptian-Arabic translation, not general assistant use. Aggressive expert removal may degrade unrelated capabilities, synthetic recovery targets may introduce bias or errors, translations can still be inaccurate or incomplete, and no additional safety evaluation was performed.

Related links

• Inference and submission code
• Paper
• Primary Egyptian Arabic k22 checkpoint
• Egyptian Arabic k24 checkpoint

Citation

@article{martin2026extracting,
  title={Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts},
  author={Martin, Liu O. and Bandarkar, Lucas and Peng, Nanyun},
  journal={arXiv preprint arXiv:2605.28042},
  year={2026}
}

License

Apache 2.0, following the base model.