---
library_name: transformers
license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
pipeline_tag: image-text-to-text
base_model:
  - google/gemma-4-31B-it-qat-q4_0-unquantized
datasets:
  - CohereLabs/aya_dataset
  - interstellarninja/hermes_reasoning_tool_use
tags:
  - quantization
  - nvfp4
---

# gemma-4-31B-it-qat-NVFP4

An NVFP4-quantized version of [google/gemma-4-31B-it-qat-q4_0-unquantized](https://huggingface.co/google/gemma-4-31B-it-qat-q4_0-unquantized).

This model was independently quantized to **NVFP4 W4A4** using **LLM Compressor** and saved in the **compressed-tensors** format.

## Model Details

* **Base model:** google/gemma-4-31B-it-qat-q4_0-unquantized
* **Architecture:** Gemma4ForConditionalGeneration
* **Quantization:** NVFP4 W4A4
* **Serialization format:** nvfp4-pack-quantized
* **Compression backend:** compressed-tensors
* **Calibration samples:** 1,024
* **Maximum calibration sequence length:** 8,192 tokens

## Calibration Data

The following datasets were used for calibration:

* [CohereLabs/aya_dataset](https://huggingface.co/datasets/CohereLabs/aya_dataset)
* [interstellarninja/hermes_reasoning_tool_use](https://huggingface.co/datasets/interstellarninja/hermes_reasoning_tool_use)

The calibration dataset consists of:

* 512 general multilingual instruction samples from Aya
* 512 reasoning and tool-use conversation samples from Hermes

## Quantization Configuration

```yaml
default_stage:
  default_modifiers:
    GPTQModifier:
      config_groups:
        group_0:
          targets: [Linear]
          weights:
            num_bits: 4
            type: float
            symmetric: true
            group_size: 16
            strategy: tensor_group
            block_structure: null
            dynamic: false
            actorder: static
            scale_dtype: torch.float8_e4m3fn
            zp_dtype: null
            observer: imatrix_mse
            observer_kwargs: {strict: true}
          input_activations:
            num_bits: 4
            type: float
            symmetric: true
            group_size: 16
            strategy: tensor_group
            block_structure: null
            dynamic: local
            actorder: null
            scale_dtype: torch.float8_e4m3fn
            zp_dtype: null
            observer: static_minmax
            observer_kwargs: {}
          output_activations: null
          format: null
      targets: [Linear]
      ignore: ['re:.*vision.*', 're:.*audio.*', lm_head, 're:.*embed.*', 're:.*router.*']
      kv_cache_scheme:
        num_bits: 8
        type: float
        symmetric: true
        group_size: null
        strategy: tensor
        block_structure: null
        dynamic: false
        actorder: null
        scale_dtype: null
        zp_dtype: null
        observer: static_minmax
        observer_kwargs: {}
      bypass_divisibility_checks: false
      requires_calibration_data: true
      block_size: 128
      dampening_frac: 0.01
      actorder: static
      offload_hessians: false
```

## Reproducibility

The quantization code and calibration preprocessing are available at:

[gemma4-nvfp4-quantization](https://github.com/yasu-oh/gemma4-nvfp4-quantization)

The quantization run used:

```text
transformers:        5.12.1
llmcompressor:       0.12.1.dev91+g4bb29d00c
compressed-tensors:  0.17.2.dev36+g2cff344
```