File size: 6,178 Bytes
4011095 de88f62 4011095 7a4d07f de88f62 7a4d07f de88f62 4011095 7a4d07f 4011095 7a4d07f 4011095 de88f62 4011095 de88f62 4011095 de88f62 7a4d07f de88f62 4011095 7a4d07f 4011095 de88f62 4011095 7a4d07f 4011095 7a4d07f 4011095 7a4d07f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 | ---
base_model: upstage/Solar-Open2-250B
base_model_relation: quantized
library_name: vllm
license: other
license_name: upstage-solar-license
license_link: LICENSE
pipeline_tag: text-generation
language:
- en
- ko
- ja
tags:
- quantization
- nvfp4
- vllm
- moe
- nota
---
# Solar Open2 250B — Nota NVFP4
[Nota AI](https://www.nota.ai/) presents a 4-bit quantized release of
[Upstage](https://www.upstage.ai/)'s
[Solar Open2 250B](https://huggingface.co/upstage/Solar-Open2-250B), produced with
Nota AI's proprietary quantization technology specialized for Mixture-of-Experts (MoE)
large language models.
## Highlights
- **NVFP4 (4-bit float, W4A4)** — `group_size=16`, packed in the `llm-compressor`
(compressed-tensors) format for direct serving in
[vLLM](https://github.com/vllm-project/vllm). Both weights and activations are
quantized to 4-bit floating point.
- > **Requires NVIDIA Blackwell.** NVFP4 relies on the FP4 tensor cores introduced in
> the Blackwell architecture (e.g. B200 / GB200), so inference must run on a
> Blackwell-class GPU. Earlier architectures (Hopper, Ada, Ampere) do not support
> NVFP4 execution.
- **Nota AI's proprietary MoE quantization framework.** This release is built upon a
suite of techniques developed by Nota AI to preserve model quality under aggressive
low-bit quantization of MoE architectures:
- A MoE-specialized calibration-dataset construction method, which achieved
**1st place across all tracks** at the NVIDIA Nemotron Hackathon.
- [**DREAM-MoE**](https://openreview.net/pdf?id=Wyhqwjl51A) and
[**SRA-MoE**](https://openreview.net/pdf?id=H0NoX02erJ), two quantization algorithms
proposed by Nota AI (published at the *ICML 2026 Workshop on AdaptFM*), which
preserve MoE routing decisions and align expert-routing behavior throughout the
quantization process.
## License
Solar Open 2 is distributed under the [**Upstage Solar License**](https://huggingface.co/upstage/Solar-Open2-250B/blob/main/LICENSE).
**Key requirements for Derivative AI Models** (create / train / fine-tune / distill / improve using Solar Open 2):
- **Naming:** prefix your model name with "Solar" (e.g., `Solar-MyModel-v1`).
- **Attribution:** prominently display "Built with Solar" in related public-facing materials.
- **Notice:** include a copy of the Upstage Solar License with your derivative model.
## Performance
### Weight footprint
| Precision | Weight footprint |
| ------------------ | :--------------: |
| BF16 | 500.6 GB |
| Nota NVFP4 | 153.3 GB |
### Benchmarks
| Benchmark | BF16 | Nota NVFP4 |
| ----------------------- | :--------------: | :---------------: |
| Tau2-Bench | 75.20 | 75.08 |
| HLE | 27.88 | 27.66 |
| GPQA Diamond | 86.26 | 85.45 |
| IFBench | 80.00 | 81.02 |
| LiveCodeBench (v5–v6) | 87.03 | 88.55 |
| MMLU-Pro | 86.19 | 86.15 |
| AIME 2026 (EN) | 95.67 | 96.67 |
| IFEval (EN) | 94.09 | 92.61 |
| HMMT | 92.05 | 90.15 |
| KMMLU-Pro | 78.38 | 77.93 |
| HAE-RAE Bench v1.1 | 73.84 | 72.84 |
| AIME (KO) | 97.67 | 97.00 |
| KBL | 75.51 | 75.40 |
| KBank-MMLU | 80.80 | 80.68 |
| KorMedMCQA | 92.99 | 93.05 |
| **Avg.** | **81.57** | **81.35** |
## Quick Start
This model is packed in the NVFP4 (compressed-tensors) format and can be served directly
with vLLM on a **Blackwell-class GPU**:
```bash
uv venv --python 3.12 --seed solar_open2_venv
source .venv/bin/activate
VLLM_PRECOMPILED_WHEEL_LOCATION="https://github.com/vllm-project/vllm/releases/download/v0.22.0/vllm-0.22.0%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl" \
VLLM_USE_PRECOMPILED=1 \
uv pip install --reinstall-package vllm --torch-backend=cu129 \
"git+https://github.com/UpstageAI/vllm.git@v0.22.0-solar-open2"
```
```bash
vllm serve nota-ai/Solar-Open2-250B-Nota-NVFP4 \
--served-model-name solar-open2-250b \
--tensor-parallel-size 4 \
--default-chat-template-kwargs '{"think_render_option":"preserved"}' \
--reasoning-parser solar_open2 \
--tool-call-parser solar_open2 \
--enable-auto-tool-choice \
--logits-processors vllm.v1.sample.logits_processor.solar_open2:SolarOpen2TemplateLogitsProcessor
```
- Set `--tensor-parallel-size` according to the number of GPUs available in your serving environment.
- See the original model card for the prompt format, parser configuration, and further details.
Send a chat completion request:
```bash
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "solar-open2-250b",
"messages": [
{"role": "user", "content": "What is Upstage?"}
],
"max_tokens": 131584,
"temperature": 1.0,
"top_p": 1.0,
"reasoning_effort": "high"
}'
```
## Citation
```bibtex
@inproceedings{park2026dreammoe,
title = {{DREAM-MoE}: Downstream Routing Error-Aware Margin-Preserving Quantization for Mixture-of-Experts Large Language Models},
author = {Park, Hancheol and Lee, Geonho and Kim, Tae-Ho},
booktitle = {ICML 2026 Workshop on Resource-Adaptive Foundation Model Inference (AdaptFM)},
year = {2026},
url = {https://openreview.net/forum?id=Wyhqwjl51A},
}
@inproceedings{lee2026sramoe,
title = {{SRA-MoE}: Output-Aware Selective Router Alignment for MoE Quantization},
author = {Lee, Geonho and Park, Hancheol and Lee, Seunghyun and Choi, Jungwook and Kim, Tae-Ho},
booktitle = {ICML 2026 Workshop on Resource-Adaptive Foundation Model Inference (AdaptFM)},
year = {2026},
url = {https://openreview.net/forum?id=H0NoX02erJ},
}
```
|