---
library_name: transformers
license: apache-2.0
base_model: Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled
tags:
  - modelopt
  - nvfp4
  - fp8-kv-cache
  - qwen3_5
  - sglang
---

# Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-ModelOpt-NVFP4

This repository contains a ModelOpt-quantized checkpoint derived from
[`Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled`](https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled).

## Quantization Summary

- Quantization tool: NVIDIA TensorRT Model Optimizer
- Weight quantization: NVFP4
- KV cache quantization: FP8
- Calibration size: 1024 samples
- Calibration sequence length: 4096
- Calibration batch size: 1
- Calibration source mix:
  - `nohurry/Opus-4.6-Reasoning-3000x-filtered`: 596 samples
  - `Jackrong/Qwen3.5-reasoning-700x`: 178 samples
  - `TeichAI/claude-4.5-opus-high-reasoning-250x`: 250 samples

The calibration set was converted into a single JSONL file with the source model's
chat template applied before PTQ, so the activation distribution is closer to the
reasoning format used by this distilled checkpoint.

## Runtime Notes

- Intended runtime target: SGLang with ModelOpt-compatible HF checkpoint loading
- Quantization format: ModelOpt HF export, not `compressed-tensors`
- A few unsupported or intentionally skipped modules may remain excluded by ModelOpt
  during export; see `hf_quant_config.json` for the final exclusion list

## Files

- `model.safetensors`: quantized weights
- `hf_quant_config.json`: final quantization metadata
- tokenizer and processor files inherited from the source checkpoint