ai-sage/GigaChat3.5-432B-A28B-Reasoning-GGUF

GigaChat 3.5 Reasoning

GigaChat 3.5 Reasoning is the first GigaChat model with full reasoning trained with online RL. Compared with GigaChat 3.5 Ultra Instruct, the largest gains are in mathematics, code, instruction following, and structured output.

This repository contains GGUF weights for llama.cpp.

Version for high-performance inference in FP8 - GigaChat3.5-432B-A28B-Reasoning.

Model in BF16 - GigaChat3.5-432B-A28B-Reasoning-bf16.

Model architecture

GigaChat 3.5 Reasoning is a 432B Mixture-of-Experts model with 28B active parameters. It uses a custom hybrid architecture that combines Multi-head Latent Attention (MLA) with GatedDeltaNet linear-attention layers.

The model also uses GatedNorm, a learned multiplicative gate applied after RMSNorm, and has three MTP heads for speculative decoding. The maximum supported context length is 262K tokens.

Online RL

Post-training starts from an SFT checkpoint. We train six domain experts independently with online RL and then combine them into one release model with on-policy distillation (OPD).

Expert	Tasks	Reward
STEM	Mathematics, olympiad problems, natural sciences	Final-answer verification
Code	Algorithms, code editing, test generation	Code execution
Code Agent	Repository-level tasks in the style of SWE-bench	Tests after applying the patch
General Agent	Function calling, user interaction, memory, search	Final environment state
Dialogue	User dialogue	Side-by-side evaluation with an LLM judge
Soft Skills	Instruction following, formats, long context, structured output	Final-answer verification

The experts are trained with CISPO. Before training, the current checkpoint is evaluated on the task pool and tasks solved in more than 75% of attempts are removed. As the model improves, the training set shifts toward harder tasks.

Rewards are domain-specific but follow the same general construction: gated checks for hard constraints, additive rewards for answer quality, and an adaptive length penalty.

After RL, the six experts are combined with on-policy distillation. The student generates its own trajectory, while the expert for the corresponding domain provides token-level supervision on that trajectory.

Benchmark scores

Task	GigaChat 3.5 Ultra Instruct	GigaChat 3.5 Ultra Reasoning	DeepSeek V4 Flash Preview Reasoning
STEM
AIME 2025, mean@32	68	89	88.95
AIME 2026, mean@32	67	92	90.4
HMMT 2025, mean@8	36.67	83.13	95.21
IMOAnswerBench*	32	73	85.75
GPQA-Diamond	61.11	82.32	87.4
General
IFBench	43.66	77	73.33
StructEval	74.35	85	80.19
MERA-2.0	24.9	42.3	--
Function Calling V4	51.57	58.59	68.06
TAU3-bench**	50.03	47.8	67.7
Natural Plan***	64	80.19	88
Code
Live Code Bench v6	56.2	85.4	87.87
SWE-bench Verified****	42.6	64.7	78.6
Terminal-Bench 2****	13.48	30.3	56.6
Arena***
Pollux	71.6	67.9	49
Arena Hard Logs V3	62.6	56.5	53.7
Arena Hard Ru	52.8	60.7	36.8
Ru LLM Arena	53.8	64	48.5
Average	51.47	68.88	72.71

* IMOAnswerBench uses Qwen-3-235B-Instruct-2507 as the judge.
** TAU3-bench is averaged across Airline, Retail, Telecom, and Banking.
*** Natural Plan uses a corrected scorer that normalizes UTF-8 characters to ASCII.
**** SWE-bench Verified and Terminal-Bench 2 use mini-swe-agent with a three-hour timeout.
***** Arena evaluations use MiniMax-M2.7 as the judge and GPT-5.2 as the baseline.

Benchmarks without a methodology-defined system prompt were evaluated with an empty system prompt.

Reasoning efficiency

On AIME 2025, AIME 2026, HMMT, and IMOAnswerBench, GigaChat 3.5 Reasoning uses 37% fewer reasoning tokens overall than DeepSeek V4 Flash Preview across the reported evaluation samples.

Task	Samples	GigaChat 3.5 Reasoning, mean tokens	DeepSeek V4 Flash Preview, mean tokens	Reduction
AIME 2025	240	13,980	19,129	27%
AIME 2026	240	13,635	17,697	23%
HMMT	480	13,311	19,553	32%
IMOAnswerBench	1,096	17,074	29,041	41%

Usage Example

Prepare the model

# 1. get the PR
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git fetch origin pull/25342/head:pr-25342
git checkout pr-25342

# 2. download the GGUF (Q8_0 shown)
pip install -U "huggingface_hub[cli]"
hf download ai-sage/GigaChat3.5-432B-A28B-Reasoning-GGUF \
  --include "GigaChat3.5-432B-A28B-Reasoning-Q8_0/*" \
  --local-dir ./gguf

GPU

Build the server:

cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-server

Start the server:

./build/bin/llama-server \
  -m ./gguf/GigaChat3.5-432B-A28B-Reasoning-Q8_0/GigaChat3.5-432B-A28B-Reasoning-Q8_0-00001-of-00010.gguf \
  -ngl 99 \
  -fa on \
  -c 32768 \
  -np 4 \
  -ctk q8_0 -ctv q8_0 \
  --jinja \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --host 0.0.0.0 --port 8080

CPU

Build the server:

cd llama.cpp
cmake -B build-cpu -DGGML_CUDA=OFF
cmake --build build-cpu --config Release -j --target llama-server

Start the server:

./build-cpu/bin/llama-server \
  -m ./gguf/GigaChat3.5-432B-A28B-Reasoning-Q8_0/GigaChat3.5-432B-A28B-Reasoning-Q8_0-00001-of-00010.gguf \
  -ngl 0 \
  -t $(nproc) \
  -fa on \
  -c 32768 \
  -np 4 \
  -ctk q8_0 -ctv q8_0 \
  --jinja \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --host 0.0.0.0 --port 8080

Request example

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ai-sage/GigaChat3.5-432B-A28B-Reasoning-Q8_0",
    "temperature": 0.6,
    "max_tokens": 2000,
    "messages": [
      {
        "role": "user",
        "content": "Докажи теорему о неподвижной точке"
      }
    ]
  }'

Function calling

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ai-sage/GigaChat3.5-432B-A28B-Reasoning-Q8_0",
    "temperature": 0.6,
    "max_tokens": 2000,
    "messages": [
      {
        "role": "user",
        "content": "Какая сейчас погода в Москве?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Получить информацию о текущей погоде в указанном городе.",
          "parameters": {
            "type": "object",
            "properties": {
              "city": {
                "type": "string",
                "description": "Название города (например, Москва, Казань)."
              }
            },
            "required": ["city"]
          }
        }
      }
    ]
  }'