File size: 1,686 Bytes
dedca19 537186e dedca19 1973da1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | ---
license: apache-2.0
library_name: onnx
pipeline_tag: text-generation
base_model: XHToken/Spark-X2.5-4B
tags:
- onnx
- webgpu
- spark2_5
- text-generation
---
# Spark-X2.5-4B-onnx
ONNX export of [XHToken/Spark-X2.5-4B](https://huggingface.co/XHToken/Spark-X2.5-4B) for onnxruntime-web / WebGPU.
Two graphs per precision under `onnx/`:
- `fp16/prefill/model.onnx` — prompt tokens -> logits + present KV
- `fp16/decode/model.onnx` — token + past KV -> logits + updated KV
Weights are ONNX external-data shards (`onnx__MatMul_*`) sitting next to each
`model.onnx` (the graphs exceed the 2 GB protobuf limit). Keep each graph's
directory together when serving.
## Architecture
spark2_5 hybrid: 3 sliding-attention layers (window 512, RoPE theta 10k) per 1 full-attention
layer (partial rotary 0.25, RoPE theta 5M). Headwise sigmoid attention output gate. GQA 16/4,
head_dim 256, tied embeddings, vocab 131072.
## Inputs
- `input_ids` [B, T] int64
- `position_ids` [B, T] int64 (absolute positions; prefill: 0..T-1; decode: past_len)
- `attn_mask` [B, T, total] additive float (0 keep / -inf mask). Host builds causal mask;
sliding layers additionally mask keys outside the 512-token window.
- `past_k_35` / `past_v_35` [B, 4, past_len, 256] — zero-length for prefill.
## Outputs
- `logits` [B, T, 131072]
- `present_k_35` / `present_v_35` [B, 4, total_len, 256]
Loop: prefill once, then feed `present_*` back as `past_*` each decode step, slicing
`position_ids` and `attn_mask` accordingly. Greedy sampling host-side from `logits`.
WebGPU: onnxruntime-web `webgpu` EP. fp16 graphs ~9 GB each; int4 weight-only graphs ~2.9 GB each — use int4 for consumer GPUs.
|