K-EXAONE-152B-A23B-REAP-80E-BF16

English

This model is an expert-pruned variant of K-EXAONE-236B-A23B, produced using Cerebras Research's REAP (Router-weighted Expert Activation Pruning) method.

The 48 lowest-saliency routed experts were removed from each of the 47 MoE layers, reducing the number of routed experts from 128 to 80 per layer.

• Total parameters: 236B → 152B
• Routed experts: 128 → 80 per MoE layer
• Active parameters: 23B, unchanged

Expert saliency was calculated as the router-weighted mean expert output norm over a calibration mixture. Shared experts, attention layers, dense layer 0, and the MTP (Multi-Token Prediction) head are fully preserved, allowing the model to retain support for self-speculative decoding.

Pruning Setup

• Method: REAP saliency, computed as mean(topk_weight * ||f_e(x)||₂) over routed tokens
• Pruning strategy: one-shot expert pruning without healing or distillation
• Experts: 128 → 80 per MoE layer across 47 layers
• Total removed experts: 48 × 47 = 2,256
• Router parameters:
  • gate.weight sliced according to retained experts
  • e_score_correction_bias sliced according to retained experts
• Routing behavior:
  • Renormalized sigmoid routing is preserved
  • Saliency statistics use the model's actual sigmoid-and-bias routing weights
  • routed_scaling_factor is included in saliency computation
• Calibration size: 16,780 samples / 101.2M tokens
• Calibration context lengths: 16K to approximately 253K tokens
• Input formatting: chat-template formatted

Calibration Mixture

Bucket	Samples	Maximum length	Purpose
evol-codealpaca	2,048	16K	Code generation
Mixture-of-Thoughts code	2,048	16K	Code reasoning
Mixture-of-Thoughts math	2,048	16K	Mathematical reasoning
Mixture-of-Thoughts science	2,048	16K	Scientific reasoning
xLAM function calling	2,048	16K	Single-turn tool calling
SWE-smith trajectories	2,048	32K	Agentic coding
Korean instruction and reasoning	3,072	16K	Preservation of Korean-specific capabilities
Korean tool and function calling	1,024	16K	Protection of tool-calling and parser behavior
Long-context QA and summarization, Korean and English	384	32K–128K	Preservation of long-context experts
256K needle stress, Korean and English	12	192K–253K	Protection of late-position routing behavior

The calibration mixture follows the Cerebras REAP agentic mixture, including code, reasoning, tool-calling, and agentic-coding data.

It is augmented with:

• Korean instruction and reasoning data
• Korean tool and function-calling data
• Korean and English long-context QA and summarization data at 32K–128K context lengths
• Korean and English needle-retrieval stress samples at 192K–253K context lengths

These additions are intended to protect Korean-specific experts, tool-calling and parser behavior, long-context experts, and routing behavior at late token positions.

One-shot-pruned MoE models may exhibit infinite reasoning loops or repetitive generation when their calibration data lacks sufficient domain and context-length diversity.

Usage

This model requires transformers >= 5.1.0, which provides native exaone_moe support.

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Baekpica/K-EXAONE-152B-A23B-REAP-80E-BF16",
    dtype="bfloat16",
    device_map="auto",
)

tokenizer = AutoTokenizer.from_pretrained(
    "Baekpica/K-EXAONE-152B-A23B-REAP-80E-BF16"
)

vLLM

Requires vLLM >= 0.14.0.

vllm serve Baekpica/K-EXAONE-152B-A23B-REAP-80E-BF16 \
  --tensor-parallel-size 4 \
  --reasoning-parser deepseek_v3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

Recommended sampling parameters:

temperature=1.0
top_p=0.95
presence_penalty=0.0

Notes and Limitations

• This model was produced using one-shot pruning without distillation or post-pruning healing.
• Some benchmarks and downstream tasks may show performance degradation relative to the original model.
• For the NVFP4-quantized variant, refer to: Baekpica/K-EXAONE-152B-A23B-REAP-80E-NVFP4
• The NVFP4 variant targets systems with approximately 128GB of unified memory, including GB10-class platforms.
• This model is distributed under the same K-EXAONE AI Model License as the original model. See the included LICENSE file.

Acknowledgements

• LG AI Research — K-EXAONE-236B-A23B
• Cerebras Research — REAP repository and REAP paper

---

K-EXAONE-152B-A23B-REAP-80E-BF16

한국어

이 모델은 **K-EXAONE-236B-A23B**에 Cerebras Research의 REAP(Router-weighted Expert Activation Pruning) 방법론을 적용하여 expert pruning한 모델입니다.

47개 MoE 레이어 각각에서 saliency가 가장 낮은 routed expert 48개를 제거하여, 레이어당 routed expert 수를 128개에서 80개로 줄였습니다.

• 총 파라미터: 236B → 152B
• Routed expert: MoE 레이어당 128개 → 80개
• 활성 파라미터: 23B 유지

Expert saliency는 calibration mixture에서 관측된 router-weighted mean expert output norm을 기준으로 계산했습니다. Shared expert, attention layer, dense layer 0, 그리고 MTP(Multi-Token Prediction) head는 모두 보존했으며, 이에 따라 self-speculative decoding을 계속 사용할 수 있습니다.

Pruning 설정

• 방법론: routed token에 대한 REAP saliency
  • mean(topk_weight * ||f_e(x)||₂)
• Pruning 방식: healing 및 distillation 없는 one-shot expert pruning
• Expert 수: 47개 MoE 레이어에서 각각 128개 → 80개
• 제거된 expert 수: 48 × 47 = 2,256개
• Router 파라미터:
  • 유지된 expert에 맞춰 gate.weight 슬라이싱
  • 유지된 expert에 맞춰 e_score_correction_bias 슬라이싱
• Routing 동작:
  • Renormalized sigmoid routing 유지
  • 모델의 실제 sigmoid 및 bias routing weight를 saliency 계산에 사용
  • routed_scaling_factor를 saliency 계산에 포함
• Calibration 규모: 16,780개 샘플 / 101.2M 토큰
• Calibration context 길이: 16K부터 약 253K 토큰
• 입력 형식: chat template 적용

Calibration 데이터 구성

버킷	샘플 수	최대 길이	목적
evol-codealpaca	2,048	16K	코드 생성
Mixture-of-Thoughts code	2,048	16K	코드 추론
Mixture-of-Thoughts math	2,048	16K	수학 추론
Mixture-of-Thoughts science	2,048	16K	과학 추론
xLAM function calling	2,048	16K	단일 턴 도구 호출
SWE-smith trajectories	2,048	32K	Agentic coding
한국어 instruction 및 reasoning	3,072	16K	한국어 특화 능력 보존
한국어 tool 및 function calling	1,024	16K	Tool calling 및 parser 동작 보호
한영 long-context QA 및 summarization	384	32K–128K	Long-context expert 보존
한영 256K needle stress	12	192K–253K	후반부 token position의 routing 보호

Calibration mixture는 코드, 추론, tool calling, agentic coding 데이터로 구성된 Cerebras REAP의 agentic mixture를 기반으로 합니다.

여기에 다음 데이터를 추가했습니다.

• 한국어 instruction 및 reasoning 데이터
• 한국어 tool 및 function-calling 데이터
• 32K–128K 길이의 한국어 및 영어 long-context QA와 summarization 데이터
• 192K–253K 길이의 한국어 및 영어 needle-retrieval stress 데이터

이러한 데이터는 한국어 특화 expert, tool calling 및 parser 동작, long-context expert, 그리고 긴 context 후반부의 routing 동작을 보호하기 위해 포함했습니다.

One-shot pruning된 MoE 모델은 calibration 데이터의 도메인 및 context 길이 다양성이 부족할 경우, 무한 reasoning loop나 반복 생성 문제가 발생할 수 있습니다.

사용 방법

이 모델은 native exaone_moe를 지원하는 transformers >= 5.1.0이 필요합니다.

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Baekpica/K-EXAONE-152B-A23B-REAP-80E-BF16",
    dtype="bfloat16",
    device_map="auto",
)

tokenizer = AutoTokenizer.from_pretrained(
    "Baekpica/K-EXAONE-152B-A23B-REAP-80E-BF16"
)

vLLM

vLLM >= 0.14.0이 필요합니다.

vllm serve Baekpica/K-EXAONE-152B-A23B-REAP-80E-BF16 \
  --tensor-parallel-size 4 \
  --reasoning-parser deepseek_v3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

권장 sampling parameter:

temperature=1.0
top_p=0.95
presence_penalty=0.0

참고사항 및 제한사항

• 이 모델은 distillation이나 pruning 이후의 healing 없이 one-shot pruning으로 제작되었습니다.
• 일부 벤치마크와 downstream task에서는 원본 모델 대비 성능 저하가 발생할 수 있습니다.
• NVFP4 양자화 버전은 다음 저장소를 참고하십시오. Baekpica/K-EXAONE-152B-A23B-REAP-80E-NVFP4
• NVFP4 버전은 GB10급 플랫폼을 포함하여 약 128GB의 unified memory를 갖춘 시스템을 대상으로 합니다.
• 라이선스는 원본 모델과 동일한 K-EXAONE AI Model License를 따릅니다. LICENSE 파일을 확인해주세요.

감사의 말

• LG AI Research — K-EXAONE-236B-A23B
• Cerebras Research — REAP 저장소 및 REAP 논문