---
license: mit
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
library_name: nxrt
tags:
  - onnx
  - onnx-genai
  - inference-metadata
---

# DeepSeek-R1-Distill-Qwen-1.5B for onnx-genai

An INT4 ONNX package that runs on CPU with the `nxrt` Python API. It
demonstrates onnx-genai's canonical `inference_metadata.yaml` format, which is
the authoritative contract in this package. `genai_config.json` is retained only
for ecosystem compatibility.

The metadata is a fully serialized `pipeline.workflow`. The autoregressive loop
is data, not runtime-specific code: the decoder graph and the ten token-policy
graphs under `policies/` are declared as workflow components, all 28 key/value
cache pairs are threaded through `serving.state_service` groups, and sampling,
termination, and length bookkeeping are ONNX graphs the runtime executes like
any other component. The package therefore runs on the generic workflow runtime
with no decoder-specific lowering step.

There is no `model.io` section. That legacy form described a single decoder in a
shape only a special-cased decoder loader could execute; it is not supported.

## Run

```bash
python3 -m pip install nxrt
ONNX_GENAI_KV_MAX_LEN=128 python3 - <<'PY'
import nxrt

engine = nxrt.genai.Engine.from_dir(".")
result = engine.generate(
    "Answer in one concise sentence: What is Rust?\nAnswer:",
    max_tokens=18,
    temperature=0.0,
)
print(result.text)
print(result.token_ids)
print(result.finish_reason)
PY
```

Verified with `nxrt==0.1.0.dev3` and `CPUExecutionProvider`:

```text
 Rust is a programming language that allows for high performance and is widely used in various industries.
[33789, 374, 264, 15473, 4128, 429, 6147, 369, 1550, 5068, 323, 374, 13570, 1483, 304, 5257, 19102, 382]
max_tokens
```

`ONNX_GENAI_KV_MAX_LEN` limits the shared KV allocation for this short example;
the model metadata preserves the full 131,072-token context limit.

## Attribution and license

Derived from
[deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
The model is distributed under the MIT License and is distilled from
[Qwen2.5-Math-1.5B](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B), which is
distributed under Apache-2.0. See `LICENSE` and the upstream model card.

<!-- inference-metadata-annotation:start -->
## Annotated inference metadata

Review [`inference_metadata.annotated.yaml`](./inference_metadata.annotated.yaml) for inline explanations of this package's workflow, tensor/state/cache contracts, and fail-closed omissions. [`inference_metadata.yaml`](./inference_metadata.yaml) remains the canonical machine-authored contract; automated validation confirms both files parse to the same metadata object.
<!-- inference-metadata-annotation:end -->